-- Logs begin at Sat 2022-06-25 15:16:57 UTC, end at Sat 2022-06-25 16:25:24 UTC. -- Jun 25 15:52:14.432043 dv-cinder-847620-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 25 15:52:14.440501 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 25 15:52:14.443054 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Jun 25 15:52:14 2022] *** Jun 25 15:52:14.443187 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 25 15:52:14.443294 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 25 15:52:14.443410 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: nodename: dv-cinder-847620-2-iscsi Jun 25 15:52:14.443410 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: machine: x86_64 Jun 25 15:52:14.443410 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: clock source: unix Jun 25 15:52:14.443410 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: pcre jit disabled Jun 25 15:52:14.443410 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: detected number of CPU cores: 4 Jun 25 15:52:14.443410 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: current working directory: / Jun 25 15:52:14.443410 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: detected binary path: /usr/bin/uwsgi-core Jun 25 15:52:14.443410 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: your processes number limit is 47680 Jun 25 15:52:14.443410 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: your memory page size is 4096 bytes Jun 25 15:52:14.443410 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: detected max file descriptor number: 2048 Jun 25 15:52:14.443410 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: lock engine: pthread robust mutexes Jun 25 15:52:14.444019 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: thunder lock: enabled Jun 25 15:52:14.444019 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 25 15:52:14.444019 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 25 15:52:14.487196 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: Python main interpreter initialized at 0x5621bf7269c0 Jun 25 15:52:14.487196 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: python threads support enabled Jun 25 15:52:14.487196 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: your server socket listen backlog is limited to 100 connections Jun 25 15:52:14.487196 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: your mercy for graceful operations on workers is 90 seconds Jun 25 15:52:14.487551 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: mapped 403077 bytes (393 KB) for 2 cores Jun 25 15:52:14.487623 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: *** Operational MODE: preforking *** Jun 25 15:52:14.487870 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: *** uWSGI is running in multiple interpreter mode *** Jun 25 15:52:14.487870 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: spawned uWSGI master process (pid: 80115) Jun 25 15:52:14.488091 dv-cinder-847620-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 25 15:52:14.489209 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: spawned uWSGI worker 1 (pid: 80124, cores: 1) Jun 25 15:52:14.489209 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: spawned uWSGI worker 2 (pid: 80125, cores: 1) Jun 25 15:52:14.489209 dv-cinder-847620-2-iscsi devstack@keystone.service[80115]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 25 15:52:19.848552 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 25 15:52:19.848552 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: from cryptography.utils import int_from_bytes Jun 25 15:52:19.849371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 25 15:52:19.849371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: from cryptography.utils import int_from_bytes Jun 25 15:52:19.853394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 25 15:52:19.853394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: from cryptography.utils import int_from_bytes Jun 25 15:52:19.854171 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 25 15:52:19.854171 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: from cryptography.utils import int_from_bytes Jun 25 15:52:19.898307 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 25 15:52:19.898472 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 25 15:52:19.898654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] command line args: [] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 25 15:52:19.898941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 25 15:52:19.899071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ================================================================================ {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 25 15:52:19.899466 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] admin_token = **** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.899562 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] config_dir = [] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.899766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.899966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] config_source = [] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.900143 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] debug = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.900307 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.902646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.902646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.902646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.902646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.902646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] list_limit = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.902646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] log_config_append = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.902646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.902646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] log_dir = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.902646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] log_file = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.903299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.903299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.903299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.903299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.903514 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.903665 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.903897 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.904092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.904360 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.904566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.904829 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.905059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.905302 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.905544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.905817 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.906302 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.906462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] publish_errors = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.906714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.906921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.907121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.907362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.907560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.907808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] standard_threads = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.908032 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.908271 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.908441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.908632 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] use_journal = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.908792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] use_json = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.908977 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] use_stderr = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.909157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] use_syslog = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.909376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.909627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.909864 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.910119 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.910342 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.910588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.910825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.911071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.911305 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.911549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.911819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.912043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.912274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.912546 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.912783 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.913026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.913234 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 25 15:52:19.913368 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.913483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.913607 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 25 15:52:19.913724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.913836 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] command line args: [] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 25 15:52:19.913965 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.914170 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 25 15:52:19.914291 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.914428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ================================================================================ {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 25 15:52:19.914542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.914675 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.914794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] admin_token = **** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.914933 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.915159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] config_dir = [] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.915296 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.915415 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.915534 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.915652 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.915764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] config_source = [] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.915880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.915992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] debug = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.916189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.916356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.916529 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.916529 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.917766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.917916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.918054 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.918176 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.918290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.918408 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.918520 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.918630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.918749 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] list_limit = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.918862 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.918978 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] log_config_append = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.919466 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.919979 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.920098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.920219 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] log_dir = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.920219 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] log_file = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.920487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.920601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.920723 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.920835 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.920954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.921073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.921184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.921300 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.921414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.921537 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.921656 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.921768 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.921879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.921988 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.922849 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.922849 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.923085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.923205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.923317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.923426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.923426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.923646 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.923757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.923873 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.923983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.923983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.924197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.924389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.924389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.924650 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.924763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.924880 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.925360 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.925648 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.925809 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.925809 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.925809 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.926117 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.926241 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.926351 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] publish_errors = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.926464 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.926464 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.926677 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.926796 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.926911 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.927030 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.927139 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.927253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.927253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.927487 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.927596 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.927596 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.927799 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.927905 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.928012 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] standard_threads = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.928126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.928240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.928348 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.928487 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.928596 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.928702 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.928827 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.928937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] use_journal = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.929044 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.929156 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] use_json = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.929264 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.929371 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] use_stderr = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.929557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.929668 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] use_syslog = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.929775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.929775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.929984 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 25 15:52:19.930091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.930206 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.930315 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.930315 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.930522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.930629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.930629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.930835 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.930927 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.931018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.931124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.931230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.931343 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.931460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.931568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.931675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.931787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.931900 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.932008 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.932122 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.932230 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.932337 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.932445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.932552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.932648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.932764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.932858 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.932950 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.933041 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.933161 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.933266 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.933359 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.933457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.933550 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.933641 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.933699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.933815 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.933926 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.934041 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.934141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.934246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.934353 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.934469 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.934586 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.934695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.934820 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.934938 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.935048 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.935148 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.935255 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.935370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] auth.external = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.935502 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.935616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.935724 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.935831 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.935939 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.936047 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.936156 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.936263 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] auth.password = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.936368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.936561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] auth.token = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.936683 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.936783 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.936889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.937004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.937113 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.937222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.937337 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.937446 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.937551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.937659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.937767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.937874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.937994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.938103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.938212 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.938320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] credential.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.938427 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.938534 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.938653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.938753 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.938860 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.938967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.939075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.939184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.939300 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.939410 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.939524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.939633 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.939741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.939741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.939959 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.940068 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.940178 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.940291 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.940401 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.940540 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.940663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.940773 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.940881 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.940989 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.941097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.941204 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.941393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.941515 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.941633 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.941829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.941965 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.942075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.942191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] federation.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.942309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.942420 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.942535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.942645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.942753 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.942861 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.942977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.942977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.943183 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.943291 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.943399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.943508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.943624 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.943734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.943842 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.943950 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.944067 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.944177 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.944294 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.944435 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.944548 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.944667 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.944777 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.944885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.945001 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.945140 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.945263 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.945373 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.945486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.945632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.945744 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.945853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.945969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.946080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.946189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.946297 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.946414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.946524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.946646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.946756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.946866 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.946974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.947081 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.947191 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.947311 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.947421 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.947530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.947639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.947748 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.947855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.947972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.948088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.948276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.948518 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.948698 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.948881 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.949064 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.949210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.949323 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.949440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.949551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.949661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.949783 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.949895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.950005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.950115 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.950230 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.950340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.950458 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.950569 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.950679 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.951099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.951099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.951309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.951427 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.951538 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.951647 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.951755 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.951865 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.951973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.952092 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.952209 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.952319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.952443 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.952554 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.952663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.952784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.952897 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.953014 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.953126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.953234 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.953344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.953461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.953571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.953679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.953787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.953895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.954004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.954124 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] auth.external = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.954241 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.954351 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.954460 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.954569 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.954695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.954877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.954991 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.955100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] auth.password = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.955208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.955316 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] auth.token = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.955425 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.955547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.955658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.955767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.955877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.955987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.956097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.956214 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.956325 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.956477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.956592 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.956702 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.956820 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.956931 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.957042 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.957159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] credential.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.957270 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.957380 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.957498 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.957609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.957720 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.957829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.957938 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.958048 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.958167 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.958278 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.958387 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.958496 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.958606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.958715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.958840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.958952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.959062 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.959172 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.959282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.959391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.959510 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.959621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.959737 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.959848 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.959958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.959958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.960171 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.960282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.960391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.960548 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.960659 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.960768 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.960889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] federation.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.960889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.961096 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.961205 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.961315 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.961424 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.961543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.961653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.961762 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.961871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.961980 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.962088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.962209 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.962329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.962440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.962549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.962657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.962766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.962884 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.962994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.963102 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.963211 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.963320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.963431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.963548 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.963659 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.963781 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.963912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.964023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.964132 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.964249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.964360 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.964489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.964599 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.964708 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.964817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.964938 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.965049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.965158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.965267 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.965375 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.965375 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.965595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.965595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.965801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] resource.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.965910 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.966019 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.966127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.966243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.966353 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.966463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.966574 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.966682 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.966790 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.966906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.967017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.967126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.967235 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.967344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.967460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.967571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.967680 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.967789 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.967897 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.968005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] role.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.968121 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.968231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] role.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.968340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.968461 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.968571 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.968681 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.968802 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.968913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.969023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.969131 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.969240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.969348 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.969464 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.969575 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.969684 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.969792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.969901 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.970015 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.970133 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.970244 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.970352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.970461 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.970569 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.970677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.970795 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.970905 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.971014 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.971122 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.971231 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.971340 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.971459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.971459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.971665 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.971775 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.971892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.972002 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.972119 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.972230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.972338 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.972468 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.972579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.972688 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.972809 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.972921 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.973030 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.973138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.973247 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.973355 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.973471 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.973584 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.973692 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.973801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.973920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.974030 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.974152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.974262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.974371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.974480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.974480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.974689 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.974807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.974919 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.975027 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.975136 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.975244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.975364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.975475 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.975584 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.975693 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.975801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.975801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] token.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.976082 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.976082 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.976292 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.976400 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.976566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.976676 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.976797 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.976908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.977033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.977181 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.977294 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.977403 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.977521 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.977632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.977741 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.977850 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.977959 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.978069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.978187 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.978298 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.978408 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.978517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.978626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.978735 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.978852 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.978962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.979072 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.979182 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.979293 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.979443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.979569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.979681 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.979790 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.979917 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.980071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.980186 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.980305 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.connection = **** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.980459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.980583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.980695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.980805 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.980914 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.981023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.981132 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.981132 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.981350 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.981460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.981568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.981678 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.981793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.981917 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.982034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.982034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.982242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.982351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.982461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.982578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.982689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.982798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.982907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.983016 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.983125 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] resource.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.983242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.983355 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.983464 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.983573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.983681 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.983798 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.983908 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.984018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.984127 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.984236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.984346 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.984481 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.984594 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80124) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 25 15:52:19.984704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.984704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.984704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.984704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] role.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.984704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] role.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.984704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.984704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.984704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.984704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.985596 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.985596 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.985596 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.985596 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.985596 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.985596 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.986205 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.986205 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.986205 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.986558 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.986743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.986974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.987218 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.987461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.987704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.987949 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.988178 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.988440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.988694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.988929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.989156 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.989391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.989621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.989866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.990116 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.990360 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.990592 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.990822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.991054 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.991285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] token.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.991517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.991759 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.991991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.992240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.992509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.992743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.992983 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.993221 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.993464 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.993695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.993928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.994158 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.994397 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.994638 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.994863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.995131 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.995427 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.995681 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.connection = **** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.995921 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.996164 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.996398 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.996658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.996890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.997121 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.997352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.997585 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.997832 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.998064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.998304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.998536 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.998778 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.999011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.999251 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.999476 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.999710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:19.999940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 25 15:52:20.000106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80125) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 25 15:52:20.012349 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.012604 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.012836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.021641 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.025977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.026206 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.026454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.035029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.049172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.049387 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.052677 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.052911 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.191684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.197381 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.240542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.240799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.253383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.253688 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.304172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.304442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.304653 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.304865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.305057 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.305268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.305645 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.305871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.306089 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.306309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.306522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.306738 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.330137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.330364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.330586 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.330709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.330848 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.330929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.331004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.331089 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.331198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.331287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.331431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.331557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.331692 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.331859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.342638 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.342924 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.355395 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.355632 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.355873 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.356091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.356214 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.356351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.356453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.356593 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.356674 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.356735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80124) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.356844 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.357052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.357295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.357503 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80125) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 25 15:52:20.374209 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.374339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.374469 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 25 15:52:20.374552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 25 15:52:20.374736 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.374860 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.374996 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 25 15:52:20.375075 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 25 15:52:20.375137 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.375253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.375380 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.375460 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.375654 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.375782 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.375919 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.376011 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.386667 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.386898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.387064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.387153 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.387222 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.387391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.387484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.387698 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.395979 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.396307 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.399243 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.399516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.399642 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.399784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.399864 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.400049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.400176 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.400313 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.400392 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.400547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.400646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.400801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.400890 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.401091 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.401217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.401394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.401958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.402096 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.402177 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.402297 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.402375 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.402375 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.428720 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.429651 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.430320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.430597 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.431237 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.431511 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.436488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.436850 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.437018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.437106 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.437195 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.437502 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.444457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.444677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.444802 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.444959 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.449304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.449563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.449687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.449805 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.449905 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.450105 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.450230 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.450364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.450441 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.450629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.450769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.450840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.450913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.451136 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.451261 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.451408 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.469960 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.470155 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.470250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.470425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.474800 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.474962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.478453 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.478577 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.478847 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.478991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.483893 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.484099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.487521 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.487646 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.491193 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.491319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.491446 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.491528 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.491752 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.491875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.492022 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.492107 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.492293 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.492441 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.492577 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.492677 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.511534 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.511739 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.511837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.512006 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.512148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.512293 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.512372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.512518 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.520426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.520723 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.522114 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.522390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.522528 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.522664 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.525568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.525838 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.525961 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.526096 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.526185 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.526466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.526728 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.526920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.527130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.527352 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.540596 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.540868 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.541158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.541436 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.543389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.543687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.543973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.546754 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.547061 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.547354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.547624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.549356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.549638 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.549906 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.561950 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.565258 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.567653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.568577 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.570298 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.571933 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.572058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.572217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.572465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.577541 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.579920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.580235 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.581535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.582863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.583265 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.583661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.584774 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.585055 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.585438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.585708 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.594881 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.595163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.595439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.595687 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.598338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.603167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.603291 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80124) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.603430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.603547 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.603837 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.603963 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.604100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.604342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.604755 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.604755 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.605071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.605301 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.605569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.605854 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.606127 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.606383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.606654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.606915 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80124) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.617437 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.617715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.617985 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.625788 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80125) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 25 15:52:20.626061 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.626335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.626602 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.626869 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.627237 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.627506 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.627766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.628030 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.628306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.628594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.628834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.629105 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.629363 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80125) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 25 15:52:20.650049 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5621bf7269c0 pid: 80124 (default app) Jun 25 15:52:20.653595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b676f-a483-446d-ba99-1ee2addd404d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:20.653888 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b676f-a483-446d-ba99-1ee2addd404d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:20.654167 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b676f-a483-446d-ba99-1ee2addd404d None None] PATH_INFO: `/v3` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:20.659116 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Sat Jun 25 15:52:20 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:20.667230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5621bf7269c0 pid: 80125 (default app) Jun 25 15:52:28.420096 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad97196-4c49-4693-8651-b6e9e7388349 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:28.420761 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad97196-4c49-4693-8651-b6e9e7388349 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:28.420761 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad97196-4c49-4693-8651-b6e9e7388349 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:28.422049 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 2/2] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:28.432117 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5574ee-78f5-4ca9-8a8d-d61925cd6be4 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:28.432819 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5574ee-78f5-4ca9-8a8d-d61925cd6be4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:28.432819 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5574ee-78f5-4ca9-8a8d-d61925cd6be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:28.503262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: INFO dbcounter [None req-df5574ee-78f5-4ca9-8a8d-d61925cd6be4 None None] Registered counter for database keystone Jun 25 15:52:28.527768 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG oslo_db.sqlalchemy.engines [None req-df5574ee-78f5-4ca9-8a8d-d61925cd6be4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80125) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 25 15:52:28.539017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writer thread running {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 25 15:52:28.732022 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG passlib.handlers.bcrypt [None req-df5574ee-78f5-4ca9-8a8d-d61925cd6be4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80125) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 25 15:52:28.732449 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG passlib.handlers.bcrypt [None req-df5574ee-78f5-4ca9-8a8d-d61925cd6be4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80125) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 25 15:52:28.787473 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-df5574ee-78f5-4ca9-8a8d-d61925cd6be4 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:28.789094 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-df5574ee-78f5-4ca9-8a8d-d61925cd6be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:28.930325 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1/3] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 503 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 25 15:52:29.093683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad97196-4c49-4693-8651-b6e9e7388349 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:29.186636 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: INFO dbcounter [None req-8ad97196-4c49-4693-8651-b6e9e7388349 None None] Registered counter for database keystone Jun 25 15:52:29.217415 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG oslo_db.sqlalchemy.engines [None req-8ad97196-4c49-4693-8651-b6e9e7388349 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80124) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 25 15:52:29.228890 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writer thread running {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 25 15:52:29.361631 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad97196-4c49-4693-8651-b6e9e7388349 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:29.367206 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c4ee0-2bef-44b4-9ddd-0ccadad4461f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:29.367873 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c4ee0-2bef-44b4-9ddd-0ccadad4461f admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:29.368154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c4ee0-2bef-44b4-9ddd-0ccadad4461f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:29.368445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c4ee0-2bef-44b4-9ddd-0ccadad4461f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:29.378243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c4ee0-2bef-44b4-9ddd-0ccadad4461f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:29.431236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: WARNING keystone.common.rbac_enforcer.enforcer [None req-603c4ee0-2bef-44b4-9ddd-0ccadad4461f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 25 15:52:29.431784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c4ee0-2bef-44b4-9ddd-0ccadad4461f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:29.442640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 3/4] 10.222.0.13 () {58 vars in 1324 bytes} [Sat Jun 25 15:52:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 25 15:52:29.449741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5574ee-78f5-4ca9-8a8d-d61925cd6be4 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:29.471009 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5574ee-78f5-4ca9-8a8d-d61925cd6be4 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:29.477521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba61eea-9d91-48a3-9a5c-2bb4dafe27bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:29.478661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba61eea-9d91-48a3-9a5c-2bb4dafe27bb admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:29.478948 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba61eea-9d91-48a3-9a5c-2bb4dafe27bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:29.479264 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba61eea-9d91-48a3-9a5c-2bb4dafe27bb admin admin] PATH_INFO: `/v3/projects/c8597e0eb76849bdadd0530f7e198ab0` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:29.483600 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba61eea-9d91-48a3-9a5c-2bb4dafe27bb admin admin] RBAC: Authorizing `identity:get_project(project_id=c8597e0eb76849bdadd0530f7e198ab0)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:29.533808 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6ba61eea-9d91-48a3-9a5c-2bb4dafe27bb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 25 15:52:29.534149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba61eea-9d91-48a3-9a5c-2bb4dafe27bb admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:29.538820 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 2/5] 10.222.0.13 () {56 vars in 1207 bytes} [Sat Jun 25 15:52:29 2022] GET /identity/v3/projects/c8597e0eb76849bdadd0530f7e198ab0 => generated 344 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:31.570233 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e74b9d-3714-47e8-bab3-4105056760ae None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:31.570820 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e74b9d-3714-47e8-bab3-4105056760ae None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:31.570820 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e74b9d-3714-47e8-bab3-4105056760ae None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:31.575010 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 4/6] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:31.582098 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e2e42-6249-4ccd-b06c-12d232700ee1 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:31.582788 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e2e42-6249-4ccd-b06c-12d232700ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:31.582921 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e2e42-6249-4ccd-b06c-12d232700ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:31.606778 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-169e2e42-6249-4ccd-b06c-12d232700ee1 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:31.608314 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-169e2e42-6249-4ccd-b06c-12d232700ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:31.618424 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 3/7] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 25 15:52:31.652945 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e74b9d-3714-47e8-bab3-4105056760ae None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:31.667299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e74b9d-3714-47e8-bab3-4105056760ae None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:31.672754 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5859162-ba2a-4072-b345-f8e5872a4d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:31.673383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5859162-ba2a-4072-b345-f8e5872a4d13 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:31.673662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5859162-ba2a-4072-b345-f8e5872a4d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:31.673917 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5859162-ba2a-4072-b345-f8e5872a4d13 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:31.682622 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5859162-ba2a-4072-b345-f8e5872a4d13 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:31.684375 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5859162-ba2a-4072-b345-f8e5872a4d13 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:31.693222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 5/8] 10.222.0.13 () {58 vars in 1324 bytes} [Sat Jun 25 15:52:31 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 25 15:52:31.699592 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e2e42-6249-4ccd-b06c-12d232700ee1 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:31.714459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e2e42-6249-4ccd-b06c-12d232700ee1 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:31.720322 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c565052-b613-4968-b005-0cd452956122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:31.721228 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c565052-b613-4968-b005-0cd452956122 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:31.721511 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c565052-b613-4968-b005-0cd452956122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:31.721783 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c565052-b613-4968-b005-0cd452956122 admin admin] PATH_INFO: `/v3/users/c132d4e24cfd4bd197338f840d4d9eb9` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:31.723358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c565052-b613-4968-b005-0cd452956122 admin admin] RBAC: Authorizing `identity:get_user(user_id=c132d4e24cfd4bd197338f840d4d9eb9)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:31.725252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c565052-b613-4968-b005-0cd452956122 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:31.726845 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 4/9] 10.222.0.13 () {56 vars in 1198 bytes} [Sat Jun 25 15:52:31 2022] GET /identity/v3/users/c132d4e24cfd4bd197338f840d4d9eb9 => generated 247 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:33.786132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406218f-8160-4579-a92f-422afe5937c9 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:33.786665 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406218f-8160-4579-a92f-422afe5937c9 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:33.786665 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406218f-8160-4579-a92f-422afe5937c9 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:33.788047 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 6/10] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:33.793411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b2bad-5315-4064-b3e3-939a0760ec67 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:33.793824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b2bad-5315-4064-b3e3-939a0760ec67 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:33.793956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b2bad-5315-4064-b3e3-939a0760ec67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:33.811336 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-543b2bad-5315-4064-b3e3-939a0760ec67 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:33.812575 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-543b2bad-5315-4064-b3e3-939a0760ec67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:33.822701 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e0da4-48d8-498d-8c24-f9fd58e248d3 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:33.823040 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e0da4-48d8-498d-8c24-f9fd58e248d3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:33.823324 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e0da4-48d8-498d-8c24-f9fd58e248d3 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:33.824822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 7/11] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:52:33.829793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf82b26-7b40-4e70-92bf-78a6828add7f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:33.830082 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf82b26-7b40-4e70-92bf-78a6828add7f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:33.830359 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf82b26-7b40-4e70-92bf-78a6828add7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:33.869993 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG passlib.handlers.bcrypt [None req-faf82b26-7b40-4e70-92bf-78a6828add7f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80124) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 25 15:52:33.870453 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG passlib.handlers.bcrypt [None req-faf82b26-7b40-4e70-92bf-78a6828add7f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80124) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 25 15:52:33.898089 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 5/12] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:52:33.917263 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-faf82b26-7b40-4e70-92bf-78a6828add7f None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:33.918845 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-faf82b26-7b40-4e70-92bf-78a6828add7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:33.921522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b2bad-5315-4064-b3e3-939a0760ec67 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:33.946119 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:34.019231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 8/13] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:52:34.029346 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b2bad-5315-4064-b3e3-939a0760ec67 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:34.040873 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf82b26-7b40-4e70-92bf-78a6828add7f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:34.064309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa897e-b089-4eb1-9d79-62f6a49a5eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:34.065209 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa897e-b089-4eb1-9d79-62f6a49a5eca None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:34.065494 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa897e-b089-4eb1-9d79-62f6a49a5eca None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:34.065774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa897e-b089-4eb1-9d79-62f6a49a5eca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:34.077971 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa897e-b089-4eb1-9d79-62f6a49a5eca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:52:34.089405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-5bfa897e-b089-4eb1-9d79-62f6a49a5eca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:52:34.089405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:52:34.089405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:52:34.089405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:52:34.089405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:52:34.089405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:52:34.089405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:52:34.089405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:52:34.089405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:52:34.089405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:52:34.089405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:52:34.089405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:52:34.089405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:52:34.089405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:52:34.089405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:52:34.089405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:52:34.090319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:52:34.090319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:52:34.090319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:52:34.090319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:52:34.090319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:52:34.090319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:52:34.090319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:52:34.090319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:52:34.090319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:52:34.090319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:52:34.090319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:52:34.090319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:52:34.090319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:52:34.090319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:52:34.090319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:52:34.090319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:52:34.091340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:52:34.091340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:52:34.091340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:52:34.091340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:52:34.091340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:52:34.091340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:52:34.091340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:52:34.091340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:52:34.091340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:52:34.091340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:52:34.091340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:52:34.091340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:52:34.091340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 6/14] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:52:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:52:34.096869 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa897e-b089-4eb1-9d79-62f6a49a5eca None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:34.140282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf82b26-7b40-4e70-92bf-78a6828add7f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:34.142867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:34.177831 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64178e-ac5b-465f-ae9d-4d26c56a3219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:34.178747 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64178e-ac5b-465f-ae9d-4d26c56a3219 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:34.179239 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64178e-ac5b-465f-ae9d-4d26c56a3219 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:34.179356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:34.179442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64178e-ac5b-465f-ae9d-4d26c56a3219 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:34.199406 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa897e-b089-4eb1-9d79-62f6a49a5eca None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:34.234089 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a64416-c4c8-4622-afb8-6a04d2976eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:34.234752 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a64416-c4c8-4622-afb8-6a04d2976eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:34.235033 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a64416-c4c8-4622-afb8-6a04d2976eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:34.235305 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a64416-c4c8-4622-afb8-6a04d2976eb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:34.236062 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a64416-c4c8-4622-afb8-6a04d2976eb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:34.236382 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a64416-c4c8-4622-afb8-6a04d2976eb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:34.237823 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a64416-c4c8-4622-afb8-6a04d2976eb0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:34.245715 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64178e-ac5b-465f-ae9d-4d26c56a3219 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:34.247232 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64178e-ac5b-465f-ae9d-4d26c56a3219 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:34.249759 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 7/15] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:52:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:34.255307 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a64416-c4c8-4622-afb8-6a04d2976eb0 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:34.345532 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 9/16] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:52:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 25 15:52:34.350926 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64178e-ac5b-465f-ae9d-4d26c56a3219 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:34.354561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:34.354711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a64416-c4c8-4622-afb8-6a04d2976eb0 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:34.387103 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eabe9e-6df7-4404-bf9a-1ed697a312fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:34.387988 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eabe9e-6df7-4404-bf9a-1ed697a312fb None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:34.388269 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eabe9e-6df7-4404-bf9a-1ed697a312fb None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:34.388579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eabe9e-6df7-4404-bf9a-1ed697a312fb None admin] PATH_INFO: `/v3/users/c132d4e24cfd4bd197338f840d4d9eb9` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:34.389813 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eabe9e-6df7-4404-bf9a-1ed697a312fb None admin] RBAC: Authorizing `identity:get_user(user_id=c132d4e24cfd4bd197338f840d4d9eb9)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:34.391292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eabe9e-6df7-4404-bf9a-1ed697a312fb None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:34.393259 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 8/17] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:52:34 2022] GET /identity/v3/users/c132d4e24cfd4bd197338f840d4d9eb9 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:52:34.398636 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eabe9e-6df7-4404-bf9a-1ed697a312fb None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:34.413324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:34.447219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64178e-ac5b-465f-ae9d-4d26c56a3219 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:34.481238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea53459-0703-4dee-9064-c89eb0d9d138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:34.482038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea53459-0703-4dee-9064-c89eb0d9d138 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:34.482318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea53459-0703-4dee-9064-c89eb0d9d138 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:34.482592 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea53459-0703-4dee-9064-c89eb0d9d138 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:34.498771 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eabe9e-6df7-4404-bf9a-1ed697a312fb None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:34.502438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea53459-0703-4dee-9064-c89eb0d9d138 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:34.503922 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea53459-0703-4dee-9064-c89eb0d9d138 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:34.525894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-1ea53459-0703-4dee-9064-c89eb0d9d138 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:52:34.525894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:52:34.525894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:52:34.525894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:52:34.525894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:52:34.525894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:52:34.525894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:52:34.525894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:52:34.525894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:52:34.525894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:52:34.525894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:52:34.525894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:52:34.525894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:52:34.525894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:52:34.525894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:52:34.527094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:52:34.527094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:52:34.527094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:52:34.527094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:52:34.527094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:52:34.527094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:52:34.527094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:52:34.527094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:52:34.527094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:52:34.527094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:52:34.527094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:52:34.527094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:52:34.527094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:52:34.527094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:52:34.527094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:52:34.527094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:52:34.527094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:52:34.528135 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:52:34.528135 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:52:34.528135 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:52:34.528135 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:52:34.528135 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 10/18] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:52:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:52:34.532961 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea53459-0703-4dee-9064-c89eb0d9d138 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:34.534306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb0d0c-0763-468b-864c-50b3545076ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:34.535104 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb0d0c-0763-468b-864c-50b3545076ce None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:34.535383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb0d0c-0763-468b-864c-50b3545076ce None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:34.535653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb0d0c-0763-468b-864c-50b3545076ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:34.536867 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb0d0c-0763-468b-864c-50b3545076ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:34.538330 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb0d0c-0763-468b-864c-50b3545076ce None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:34.539932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 9/19] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:52:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:34.545757 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb0d0c-0763-468b-864c-50b3545076ce None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:34.592125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:34.625343 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:34.635884 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea53459-0703-4dee-9064-c89eb0d9d138 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:34.645069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb0d0c-0763-468b-864c-50b3545076ce None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:34.669370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ca763-b8bb-4927-ae96-e874bb7a8b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:34.670036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ca763-b8bb-4927-ae96-e874bb7a8b24 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:34.670300 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ca763-b8bb-4927-ae96-e874bb7a8b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:34.670575 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ca763-b8bb-4927-ae96-e874bb7a8b24 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:34.671303 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ca763-b8bb-4927-ae96-e874bb7a8b24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:34.671608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ca763-b8bb-4927-ae96-e874bb7a8b24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:34.673047 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ca763-b8bb-4927-ae96-e874bb7a8b24 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:34.679506 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a244aa-5e70-4564-9a1c-b21be145049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:34.680158 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a244aa-5e70-4564-9a1c-b21be145049b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:34.680441 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a244aa-5e70-4564-9a1c-b21be145049b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:34.680732 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a244aa-5e70-4564-9a1c-b21be145049b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:34.681555 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a244aa-5e70-4564-9a1c-b21be145049b None admin] RBAC: Adding query filter params (role.id=b1e13c449fa64b3e88b6832bb767214b, scope.domain.id=default, user.id=c132d4e24cfd4bd197338f840d4d9eb9) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:34.681859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a244aa-5e70-4564-9a1c-b21be145049b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:34.683538 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a244aa-5e70-4564-9a1c-b21be145049b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:34.690521 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 11/20] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:34.690669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 10/21] 10.222.0.13 () {56 vars in 1322 bytes} [Sat Jun 25 15:52:34 2022] GET /identity/v3/role_assignments?user.id=c132d4e24cfd4bd197338f840d4d9eb9&scope.domain.id=default&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:52:36.847583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c58f92-31de-46b5-94c8-0cbdf56c0751 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:36.848096 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c58f92-31de-46b5-94c8-0cbdf56c0751 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:36.848167 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c58f92-31de-46b5-94c8-0cbdf56c0751 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:36.849538 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 12/22] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:36.855147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd2ae4-a396-4cd4-ab9c-b952af700103 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:36.855546 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd2ae4-a396-4cd4-ab9c-b952af700103 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:36.855742 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd2ae4-a396-4cd4-ab9c-b952af700103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:36.885488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b1bd2ae4-a396-4cd4-ab9c-b952af700103 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:36.886978 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b1bd2ae4-a396-4cd4-ab9c-b952af700103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:36.974678 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 11/23] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:52:37.009479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c58f92-31de-46b5-94c8-0cbdf56c0751 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:37.153873 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c58f92-31de-46b5-94c8-0cbdf56c0751 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:37.170101 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e7a4d-bd7e-41a6-98dd-e15a648098ab None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:37.170388 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e7a4d-bd7e-41a6-98dd-e15a648098ab None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:37.170661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e7a4d-bd7e-41a6-98dd-e15a648098ab None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:37.172201 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 12/24] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:37.177515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:37.184747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891313a-ac8d-4860-8cef-9073018948af None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:37.185040 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891313a-ac8d-4860-8cef-9073018948af None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:37.185304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891313a-ac8d-4860-8cef-9073018948af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:37.201375 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fcf49d-a270-4196-bb78-a853239b09d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:37.202191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcf49d-a270-4196-bb78-a853239b09d5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:37.202475 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcf49d-a270-4196-bb78-a853239b09d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:37.202754 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcf49d-a270-4196-bb78-a853239b09d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:37.221443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-6891313a-ac8d-4860-8cef-9073018948af None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:37.222658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6891313a-ac8d-4860-8cef-9073018948af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:37.226411 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fcf49d-a270-4196-bb78-a853239b09d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:52:37.235746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-18fcf49d-a270-4196-bb78-a853239b09d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:52:37.235746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:52:37.235746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:52:37.235746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:52:37.235746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:52:37.235746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:52:37.235746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:52:37.235746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:52:37.235746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:52:37.235746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:52:37.235746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:52:37.235746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:52:37.235746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:52:37.235746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:52:37.235746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:52:37.235746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:52:37.236668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:52:37.236668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:52:37.236668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:52:37.236668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:52:37.236668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:52:37.236668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:52:37.236668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:52:37.236668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:52:37.236668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:52:37.236668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:52:37.236668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:52:37.236668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:52:37.236668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:52:37.236668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:52:37.236668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:52:37.236668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:52:37.237421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:52:37.237421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:52:37.237421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:52:37.237421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:52:37.237421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:52:37.237421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:52:37.237421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:52:37.237421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:52:37.237421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:52:37.237421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:52:37.237421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:52:37.237421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:52:37.237421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 13/25] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:52:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:52:37.238651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafc090-9dd1-40b9-8efb-1d1ec0c5edd8 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:37.238651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafc090-9dd1-40b9-8efb-1d1ec0c5edd8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:37.238924 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafc090-9dd1-40b9-8efb-1d1ec0c5edd8 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:37.240193 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 14/26] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:37.244809 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafc090-9dd1-40b9-8efb-1d1ec0c5edd8 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:37.352031 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 13/27] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:52:37.353767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa8133-9201-4121-8740-615e791ffb4f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:37.354862 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa8133-9201-4121-8740-615e791ffb4f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:37.357399 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa8133-9201-4121-8740-615e791ffb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:37.366921 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:37.388442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-bcfa8133-9201-4121-8740-615e791ffb4f None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:37.389707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bcfa8133-9201-4121-8740-615e791ffb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:37.415585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafc090-9dd1-40b9-8efb-1d1ec0c5edd8 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:37.483211 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b2331-c037-469a-8063-b70c14606b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:37.483890 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b2331-c037-469a-8063-b70c14606b4b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:37.484170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b2331-c037-469a-8063-b70c14606b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:37.484466 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b2331-c037-469a-8063-b70c14606b4b None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:37.485232 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b2331-c037-469a-8063-b70c14606b4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:37.485539 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b2331-c037-469a-8063-b70c14606b4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:37.486939 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b2331-c037-469a-8063-b70c14606b4b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:37.504989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 15/28] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:52:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:37.505807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b2331-c037-469a-8063-b70c14606b4b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:37.514590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 14/29] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:52:37.515437 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa8133-9201-4121-8740-615e791ffb4f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:37.573062 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:37.670654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b2331-c037-469a-8063-b70c14606b4b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:37.670814 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa8133-9201-4121-8740-615e791ffb4f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:37.706980 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4178f-db49-4423-9ce9-4079ab168b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:37.707664 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4178f-db49-4423-9ce9-4079ab168b65 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:37.707947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4178f-db49-4423-9ce9-4079ab168b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:37.708224 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4178f-db49-4423-9ce9-4079ab168b65 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:37.708436 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce751fdf-b6bb-4599-817f-d9db28748c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:37.709017 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4178f-db49-4423-9ce9-4079ab168b65 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:37.709322 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce751fdf-b6bb-4599-817f-d9db28748c59 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:37.709601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce751fdf-b6bb-4599-817f-d9db28748c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:37.709872 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce751fdf-b6bb-4599-817f-d9db28748c59 None admin] PATH_INFO: `/v3/users/c132d4e24cfd4bd197338f840d4d9eb9` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:37.710413 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4178f-db49-4423-9ce9-4079ab168b65 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:37.711144 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce751fdf-b6bb-4599-817f-d9db28748c59 None admin] RBAC: Authorizing `identity:get_user(user_id=c132d4e24cfd4bd197338f840d4d9eb9)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:37.712692 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce751fdf-b6bb-4599-817f-d9db28748c59 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:37.714797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 15/30] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:52:37 2022] GET /identity/v3/users/c132d4e24cfd4bd197338f840d4d9eb9 => generated 247 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:37.715700 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce751fdf-b6bb-4599-817f-d9db28748c59 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:37.718875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:37.820940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce751fdf-b6bb-4599-817f-d9db28748c59 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:37.848375 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 16/31] 10.222.0.13 () {60 vars in 1131 bytes} [Sat Jun 25 15:52:37 2022] POST /identity/v3/roles => generated 225 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:52:37.853287 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e81f3-bf94-42fa-859a-f77704171015 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:37.853772 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e81f3-bf94-42fa-859a-f77704171015 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:37.854300 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e81f3-bf94-42fa-859a-f77704171015 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:37.855383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ea7a4-df87-42ec-a4ee-704bc7fe4bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:37.855928 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 17/32] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:37.856957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ea7a4-df87-42ec-a4ee-704bc7fe4bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:37.857446 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ea7a4-df87-42ec-a4ee-704bc7fe4bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:37.857927 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ea7a4-df87-42ec-a4ee-704bc7fe4bd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:37.860852 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e81f3-bf94-42fa-859a-f77704171015 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:37.871972 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ea7a4-df87-42ec-a4ee-704bc7fe4bd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:37.873811 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ea7a4-df87-42ec-a4ee-704bc7fe4bd0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:37.894001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-259ea7a4-df87-42ec-a4ee-704bc7fe4bd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:52:37.894001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:52:37.894001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:52:37.894001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:52:37.894001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:52:37.894001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:52:37.894001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:52:37.894001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:52:37.894001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:52:37.894001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:52:37.894001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:52:37.894001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:52:37.894001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:52:37.894001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:52:37.894001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:52:37.894866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:52:37.894866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:52:37.894866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:52:37.894866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:52:37.894866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:52:37.894866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:52:37.894866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:52:37.894866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:52:37.894866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:52:37.894866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:52:37.894866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:52:37.894866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:52:37.894866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:52:37.894866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:52:37.894866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:52:37.894866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:52:37.894866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:52:37.895640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:52:37.895640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:52:37.895640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:52:37.895640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:52:37.897856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 16/33] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:52:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:52:37.899476 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c81c7-4407-43a0-b755-20b4c8ac3ddf None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:37.899948 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c81c7-4407-43a0-b755-20b4c8ac3ddf None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:37.900429 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c81c7-4407-43a0-b755-20b4c8ac3ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:37.919771 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a54c81c7-4407-43a0-b755-20b4c8ac3ddf None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:37.921482 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a54c81c7-4407-43a0-b755-20b4c8ac3ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:37.988815 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:38.005149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 17/34] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:52:38.006334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c81c7-4407-43a0-b755-20b4c8ac3ddf None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:38.084011 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:38.101134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e81f3-bf94-42fa-859a-f77704171015 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:38.130197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c81c7-4407-43a0-b755-20b4c8ac3ddf None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:38.172684 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7984ce7-5945-43df-aa9f-2cbb50395e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:38.173727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7984ce7-5945-43df-aa9f-2cbb50395e5d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:38.173882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8833e19-f4cc-4de7-965f-1a6ff75d56be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:38.174290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7984ce7-5945-43df-aa9f-2cbb50395e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:38.174702 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7984ce7-5945-43df-aa9f-2cbb50395e5d None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:38.175202 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8833e19-f4cc-4de7-965f-1a6ff75d56be None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:38.175626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8833e19-f4cc-4de7-965f-1a6ff75d56be None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:38.175742 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7984ce7-5945-43df-aa9f-2cbb50395e5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:38.176092 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7984ce7-5945-43df-aa9f-2cbb50395e5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:38.176278 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8833e19-f4cc-4de7-965f-1a6ff75d56be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:38.177728 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7984ce7-5945-43df-aa9f-2cbb50395e5d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:38.178163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8833e19-f4cc-4de7-965f-1a6ff75d56be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:38.179952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8833e19-f4cc-4de7-965f-1a6ff75d56be None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:38.182247 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 18/35] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:52:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:38.183220 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8833e19-f4cc-4de7-965f-1a6ff75d56be None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:38.201313 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 18/36] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:38.202200 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7984ce7-5945-43df-aa9f-2cbb50395e5d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:38.259446 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:38.315221 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8833e19-f4cc-4de7-965f-1a6ff75d56be None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:38.321909 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7984ce7-5945-43df-aa9f-2cbb50395e5d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:38.364723 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670ad46-9de1-461f-a2c5-fe1aa2c24cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:38.365415 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670ad46-9de1-461f-a2c5-fe1aa2c24cb0 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:38.365700 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670ad46-9de1-461f-a2c5-fe1aa2c24cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:38.365978 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670ad46-9de1-461f-a2c5-fe1aa2c24cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:38.367496 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670ad46-9de1-461f-a2c5-fe1aa2c24cb0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:38.368128 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670ad46-9de1-461f-a2c5-fe1aa2c24cb0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:38.372078 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82a29b-602a-416c-8218-035d14aa1370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:38.372989 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82a29b-602a-416c-8218-035d14aa1370 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:38.373264 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82a29b-602a-416c-8218-035d14aa1370 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:38.373541 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82a29b-602a-416c-8218-035d14aa1370 None admin] PATH_INFO: `/v3/domains/default/users/c132d4e24cfd4bd197338f840d4d9eb9/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:38.375816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f82a29b-602a-416c-8218-035d14aa1370 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c132d4e24cfd4bd197338f840d4d9eb9, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:38.377309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f82a29b-602a-416c-8218-035d14aa1370 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:38.400562 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 19/37] 10.222.0.13 () {60 vars in 1131 bytes} [Sat Jun 25 15:52:38 2022] POST /identity/v3/roles => generated 219 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:52:38.401897 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670ad46-9de1-461f-a2c5-fe1aa2c24cb0 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:38.452023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:38.516741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 19/38] 10.222.0.13 () {58 vars in 1361 bytes} [Sat Jun 25 15:52:38 2022] PUT /identity/v3/domains/default/users/c132d4e24cfd4bd197338f840d4d9eb9/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:52:38.516741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f6297-6f42-47f1-8133-eb493d1d1545 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:38.516741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f6297-6f42-47f1-8133-eb493d1d1545 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:38.516741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f6297-6f42-47f1-8133-eb493d1d1545 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:38.518469 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 20/39] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:38.530250 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662feb1d-5983-4852-9b93-03b05c520990 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:38.530536 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662feb1d-5983-4852-9b93-03b05c520990 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:38.530817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662feb1d-5983-4852-9b93-03b05c520990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:38.557664 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670ad46-9de1-461f-a2c5-fe1aa2c24cb0 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:38.599615 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678bc3c-3df1-40a1-a4f8-2a622c85661b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:38.600604 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678bc3c-3df1-40a1-a4f8-2a622c85661b None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:38.601070 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678bc3c-3df1-40a1-a4f8-2a622c85661b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:38.601493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678bc3c-3df1-40a1-a4f8-2a622c85661b None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:38.602404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678bc3c-3df1-40a1-a4f8-2a622c85661b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:38.604575 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678bc3c-3df1-40a1-a4f8-2a622c85661b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:38.648733 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 20/40] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Jun 25 15:52:38 2022] POST /identity/v3/projects => generated 301 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:52:38.686247 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a73b0a-8bb7-4b1c-9c87-5880f62deb5c None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:38.686576 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a73b0a-8bb7-4b1c-9c87-5880f62deb5c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:38.686827 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a73b0a-8bb7-4b1c-9c87-5880f62deb5c None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:38.688451 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 21/41] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:38.694480 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd690de-b52e-4e1d-a456-8d165f251354 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:38.695099 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd690de-b52e-4e1d-a456-8d165f251354 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:38.695639 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd690de-b52e-4e1d-a456-8d165f251354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:38.712068 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-662feb1d-5983-4852-9b93-03b05c520990 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:38.713844 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-662feb1d-5983-4852-9b93-03b05c520990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:38.717132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3dd690de-b52e-4e1d-a456-8d165f251354 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:38.719209 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3dd690de-b52e-4e1d-a456-8d165f251354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:38.812887 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 22/42] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:52:38.819097 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 21/43] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:52:38.835920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd690de-b52e-4e1d-a456-8d165f251354 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:38.843147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662feb1d-5983-4852-9b93-03b05c520990 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:38.846850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:38.882668 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:38.954659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd690de-b52e-4e1d-a456-8d165f251354 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:38.968153 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662feb1d-5983-4852-9b93-03b05c520990 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:38.989327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49388916-7e6e-4344-851b-133e09899396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:38.990776 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49388916-7e6e-4344-851b-133e09899396 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:38.991202 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49388916-7e6e-4344-851b-133e09899396 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:38.991879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49388916-7e6e-4344-851b-133e09899396 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:38.993422 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49388916-7e6e-4344-851b-133e09899396 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:38.996503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49388916-7e6e-4344-851b-133e09899396 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:38.999095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 23/44] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:52:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:39.004447 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49388916-7e6e-4344-851b-133e09899396 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:39.007168 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097732b3-180d-4ac5-b422-3773aeed4297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:39.008502 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097732b3-180d-4ac5-b422-3773aeed4297 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:39.008823 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097732b3-180d-4ac5-b422-3773aeed4297 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:39.009106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097732b3-180d-4ac5-b422-3773aeed4297 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:39.009884 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097732b3-180d-4ac5-b422-3773aeed4297 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:39.011650 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097732b3-180d-4ac5-b422-3773aeed4297 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:39.028827 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 22/45] 10.222.0.13 () {60 vars in 1131 bytes} [Sat Jun 25 15:52:38 2022] POST /identity/v3/roles => generated 223 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 15:52:39.101854 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:39.128679 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49388916-7e6e-4344-851b-133e09899396 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:39.165917 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd7c8a-9db6-4de7-a1b5-d3c3e40a14ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:39.166707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd7c8a-9db6-4de7-a1b5-d3c3e40a14ce None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:39.166995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd7c8a-9db6-4de7-a1b5-d3c3e40a14ce None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:39.167326 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd7c8a-9db6-4de7-a1b5-d3c3e40a14ce None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:39.168136 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd7c8a-9db6-4de7-a1b5-d3c3e40a14ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:39.169684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd7c8a-9db6-4de7-a1b5-d3c3e40a14ce None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:39.198549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 24/46] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Jun 25 15:52:39 2022] POST /identity/v3/projects => generated 312 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 15:52:40.640048 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59cd8a-682b-4a0a-a666-741565aba8ca None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:40.640587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59cd8a-682b-4a0a-a666-741565aba8ca None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:40.640667 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59cd8a-682b-4a0a-a666-741565aba8ca None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:40.642348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 23/47] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:40.647834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc7b53-4289-480f-a6a7-d9d89f9c0029 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:40.648205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc7b53-4289-480f-a6a7-d9d89f9c0029 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:40.648399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc7b53-4289-480f-a6a7-d9d89f9c0029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:40.665892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7ccc7b53-4289-480f-a6a7-d9d89f9c0029 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:40.667187 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7ccc7b53-4289-480f-a6a7-d9d89f9c0029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:40.736240 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 25/48] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:52:40.759288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59cd8a-682b-4a0a-a666-741565aba8ca None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:40.864564 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:40.874792 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59cd8a-682b-4a0a-a666-741565aba8ca None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:40.910483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00f0bd-66bc-4693-9996-1141b73a4ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:40.911205 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00f0bd-66bc-4693-9996-1141b73a4ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:40.911484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00f0bd-66bc-4693-9996-1141b73a4ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:40.911753 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00f0bd-66bc-4693-9996-1141b73a4ae8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:40.923768 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00f0bd-66bc-4693-9996-1141b73a4ae8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:52:40.931272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-bb00f0bd-66bc-4693-9996-1141b73a4ae8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:52:40.931272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:52:40.931272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:52:40.931272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:52:40.931272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:52:40.931272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:52:40.931272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:52:40.931272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:52:40.931272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:52:40.931272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:52:40.931272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:52:40.931272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:52:40.931272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:52:40.931272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:52:40.931272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:52:40.931272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:52:40.932621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:52:40.932621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:52:40.932621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:52:40.932621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:52:40.932621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:52:40.932621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:52:40.932621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:52:40.932621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:52:40.932621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:52:40.932621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:52:40.932621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:52:40.932621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:52:40.932621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:52:40.932621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:52:40.932621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:52:40.932621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:52:40.933462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:52:40.933462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:52:40.933462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:52:40.933462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:52:40.933462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:52:40.933462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:52:40.933462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:52:40.933462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:52:40.933462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:52:40.933462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:52:40.933462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:52:40.933462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:52:40.933462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 24/49] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:52:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:52:40.938565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc7b53-4289-480f-a6a7-d9d89f9c0029 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:41.000017 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:41.043793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc7b53-4289-480f-a6a7-d9d89f9c0029 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:41.081037 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d5523-f749-4fe1-b7e2-347a52415ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:41.081720 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d5523-f749-4fe1-b7e2-347a52415ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:41.082006 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d5523-f749-4fe1-b7e2-347a52415ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:41.082288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d5523-f749-4fe1-b7e2-347a52415ec0 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:41.083051 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d5523-f749-4fe1-b7e2-347a52415ec0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:41.083361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d5523-f749-4fe1-b7e2-347a52415ec0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:41.084797 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d5523-f749-4fe1-b7e2-347a52415ec0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:41.093719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 26/50] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:52:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:41.099364 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00f0bd-66bc-4693-9996-1141b73a4ae8 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:41.196423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5671a-7ae5-454c-b495-2bc87c136623 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:41.196735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5671a-7ae5-454c-b495-2bc87c136623 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:41.197024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5671a-7ae5-454c-b495-2bc87c136623 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:41.199166 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 27/51] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:41.202435 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00f0bd-66bc-4693-9996-1141b73a4ae8 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:41.204252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89bc76-85a6-45aa-87b5-e0225f41d569 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:41.204584 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89bc76-85a6-45aa-87b5-e0225f41d569 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:41.204869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89bc76-85a6-45aa-87b5-e0225f41d569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:41.237959 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bff379-92b2-4da1-a960-55f921dc9a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:41.238846 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bff379-92b2-4da1-a960-55f921dc9a18 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:41.239129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bff379-92b2-4da1-a960-55f921dc9a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:41.239400 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bff379-92b2-4da1-a960-55f921dc9a18 None admin] PATH_INFO: `/v3/users/c132d4e24cfd4bd197338f840d4d9eb9` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:41.240849 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bff379-92b2-4da1-a960-55f921dc9a18 None admin] RBAC: Authorizing `identity:get_user(user_id=c132d4e24cfd4bd197338f840d4d9eb9)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:41.242366 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bff379-92b2-4da1-a960-55f921dc9a18 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:41.244080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 25/52] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:52:41 2022] GET /identity/v3/users/c132d4e24cfd4bd197338f840d4d9eb9 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:41.250318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bff379-92b2-4da1-a960-55f921dc9a18 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:41.278208 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:41.329566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ac89bc76-85a6-45aa-87b5-e0225f41d569 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:41.330800 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ac89bc76-85a6-45aa-87b5-e0225f41d569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:41.356265 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bff379-92b2-4da1-a960-55f921dc9a18 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:41.390536 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60644b84-b6b0-4f36-85b7-0d3e8a35b8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:41.391325 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60644b84-b6b0-4f36-85b7-0d3e8a35b8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:41.391915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60644b84-b6b0-4f36-85b7-0d3e8a35b8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:41.392185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60644b84-b6b0-4f36-85b7-0d3e8a35b8a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:41.393385 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60644b84-b6b0-4f36-85b7-0d3e8a35b8a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:41.394842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60644b84-b6b0-4f36-85b7-0d3e8a35b8a2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:41.396633 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 26/53] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:52:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:52:41.400247 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 28/54] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:52:41.402223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60644b84-b6b0-4f36-85b7-0d3e8a35b8a2 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:41.424060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac89bc76-85a6-45aa-87b5-e0225f41d569 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:41.472154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:41.505551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:41.515419 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60644b84-b6b0-4f36-85b7-0d3e8a35b8a2 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:41.523997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac89bc76-85a6-45aa-87b5-e0225f41d569 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:41.549095 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0f9b6-7bd2-4704-aed0-62d45a30cbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:41.549750 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0f9b6-7bd2-4704-aed0-62d45a30cbe9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:41.550028 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0f9b6-7bd2-4704-aed0-62d45a30cbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:41.550307 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0f9b6-7bd2-4704-aed0-62d45a30cbe9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:41.551136 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0f9b6-7bd2-4704-aed0-62d45a30cbe9 None admin] RBAC: Adding query filter params (role.id=b1e13c449fa64b3e88b6832bb767214b, scope.domain.id=default, user.id=c132d4e24cfd4bd197338f840d4d9eb9) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:41.551445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0f9b6-7bd2-4704-aed0-62d45a30cbe9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:41.552939 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0f9b6-7bd2-4704-aed0-62d45a30cbe9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:41.558056 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6414f985-096c-41f6-b345-42c542bc3a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:41.558863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 27/55] 10.222.0.13 () {56 vars in 1322 bytes} [Sat Jun 25 15:52:41 2022] GET /identity/v3/role_assignments?user.id=c132d4e24cfd4bd197338f840d4d9eb9&scope.domain.id=default&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 530 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:52:41.558982 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414f985-096c-41f6-b345-42c542bc3a42 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:41.559126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414f985-096c-41f6-b345-42c542bc3a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:41.559400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414f985-096c-41f6-b345-42c542bc3a42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:41.560640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6414f985-096c-41f6-b345-42c542bc3a42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:41.562087 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6414f985-096c-41f6-b345-42c542bc3a42 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:41.563877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 29/56] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:52:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:52:41.569686 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0f9b6-7bd2-4704-aed0-62d45a30cbe9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:41.682172 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0f9b6-7bd2-4704-aed0-62d45a30cbe9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:41.718405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde33475-c2af-4851-96e4-a8df7ffa7dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:41.719067 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde33475-c2af-4851-96e4-a8df7ffa7dbf None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:41.719351 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde33475-c2af-4851-96e4-a8df7ffa7dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:41.719606 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde33475-c2af-4851-96e4-a8df7ffa7dbf None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:41.720339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde33475-c2af-4851-96e4-a8df7ffa7dbf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:41.721809 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde33475-c2af-4851-96e4-a8df7ffa7dbf None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:41.752287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 28/57] 10.222.0.13 () {60 vars in 1140 bytes} [Sat Jun 25 15:52:41 2022] POST /identity/v3/projects => generated 298 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:52:43.774998 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816ecec-e5d5-4648-bb75-bd36a2fedfe7 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:43.775839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816ecec-e5d5-4648-bb75-bd36a2fedfe7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:43.775977 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816ecec-e5d5-4648-bb75-bd36a2fedfe7 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:43.778141 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 30/58] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:43.786716 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359173f-3768-4267-9515-930a248ad368 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:43.787114 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359173f-3768-4267-9515-930a248ad368 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:43.787367 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359173f-3768-4267-9515-930a248ad368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:43.805857 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:43.813290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e359173f-3768-4267-9515-930a248ad368 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:43.815067 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e359173f-3768-4267-9515-930a248ad368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:43.888288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 29/59] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:52:43.924143 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816ecec-e5d5-4648-bb75-bd36a2fedfe7 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:44.033714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816ecec-e5d5-4648-bb75-bd36a2fedfe7 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:44.051098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:44.079218 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad418d64-dbe2-48f2-a086-fb440271a481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:44.080011 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad418d64-dbe2-48f2-a086-fb440271a481 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:44.080293 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad418d64-dbe2-48f2-a086-fb440271a481 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:44.080621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad418d64-dbe2-48f2-a086-fb440271a481 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:44.081863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad418d64-dbe2-48f2-a086-fb440271a481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:44.083316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad418d64-dbe2-48f2-a086-fb440271a481 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:44.084976 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 31/60] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:52:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:44.090763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359173f-3768-4267-9515-930a248ad368 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:44.194968 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359173f-3768-4267-9515-930a248ad368 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:44.229892 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07130785-5904-421e-a55a-93100ea26170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:44.230547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07130785-5904-421e-a55a-93100ea26170 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:44.230832 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07130785-5904-421e-a55a-93100ea26170 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:44.231103 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07130785-5904-421e-a55a-93100ea26170 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:44.231814 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07130785-5904-421e-a55a-93100ea26170 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:44.233313 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07130785-5904-421e-a55a-93100ea26170 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:44.243754 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:44.261000 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 30/61] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Jun 25 15:52:44 2022] POST /identity/v3/users => generated 275 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:52:47.535624 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b58fd-1d3b-4538-884e-caae089834d4 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:47.536531 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b58fd-1d3b-4538-884e-caae089834d4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:47.536948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b58fd-1d3b-4538-884e-caae089834d4 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:47.538695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 32/62] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:47.544729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6404201-394b-4d12-a177-d53521631c06 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:47.545573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6404201-394b-4d12-a177-d53521631c06 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:47.545999 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6404201-394b-4d12-a177-d53521631c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:47.573586 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a6404201-394b-4d12-a177-d53521631c06 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:47.575096 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a6404201-394b-4d12-a177-d53521631c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:47.586020 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d19db-9dc7-40da-9400-c0d37df2e108 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:47.586513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d19db-9dc7-40da-9400-c0d37df2e108 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:47.586950 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d19db-9dc7-40da-9400-c0d37df2e108 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:47.588722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 33/63] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:47.594702 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995f06e-ab34-4947-a553-00bc1d9be256 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:47.595172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995f06e-ab34-4947-a553-00bc1d9be256 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:47.595586 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995f06e-ab34-4947-a553-00bc1d9be256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:47.617641 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-2995f06e-ab34-4947-a553-00bc1d9be256 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:47.619054 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2995f06e-ab34-4947-a553-00bc1d9be256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:47.721896 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 31/64] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:52:47.722400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 34/65] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:52:47.745398 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6404201-394b-4d12-a177-d53521631c06 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:47.757700 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995f06e-ab34-4947-a553-00bc1d9be256 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:47.854490 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6404201-394b-4d12-a177-d53521631c06 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:47.871220 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995f06e-ab34-4947-a553-00bc1d9be256 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:47.884380 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:47.890187 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:47.900882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8742b8-b922-42e1-bc13-3515b1fdbdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:47.902155 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8742b8-b922-42e1-bc13-3515b1fdbdda None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:47.902458 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8742b8-b922-42e1-bc13-3515b1fdbdda None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:47.902734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8742b8-b922-42e1-bc13-3515b1fdbdda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:47.910059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2932c-e4a4-4676-b24b-a168596b217e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:47.910865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2932c-e4a4-4676-b24b-a168596b217e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:47.911151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2932c-e4a4-4676-b24b-a168596b217e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:47.911541 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2932c-e4a4-4676-b24b-a168596b217e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:47.914316 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8742b8-b922-42e1-bc13-3515b1fdbdda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:52:47.922001 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2932c-e4a4-4676-b24b-a168596b217e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:52:47.922301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-bb8742b8-b922-42e1-bc13-3515b1fdbdda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:52:47.922301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:52:47.922301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:52:47.922301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:52:47.922301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:52:47.922301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:52:47.922301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:52:47.922301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:52:47.922301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:52:47.922301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:52:47.922301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:52:47.922301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:52:47.922301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:52:47.922301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:52:47.922301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:52:47.922301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:52:47.923167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:52:47.923167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:52:47.923167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:52:47.923167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:52:47.923167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:52:47.923167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:52:47.923167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:52:47.923167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:52:47.923167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:52:47.923167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:52:47.923167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:52:47.923167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:52:47.923167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:52:47.923167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:52:47.923167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:52:47.923167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:52:47.923907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:52:47.923907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:52:47.923907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:52:47.923907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:52:47.923907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:52:47.923907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:52:47.923907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:52:47.923907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:52:47.923907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:52:47.923907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:52:47.923907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:52:47.923907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:52:47.923907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 32/66] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:52:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:52:47.925170 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3f0e4-5c48-4ddd-9edf-7c07be859252 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:47.925430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3f0e4-5c48-4ddd-9edf-7c07be859252 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:47.925711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3f0e4-5c48-4ddd-9edf-7c07be859252 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:47.927133 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 33/67] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:47.928307 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0747b-a836-4810-99cc-545e26ea9327 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:47.928615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0747b-a836-4810-99cc-545e26ea9327 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:47.928889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0747b-a836-4810-99cc-545e26ea9327 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:47.932555 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 34/68] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:47.932555 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0747b-a836-4810-99cc-545e26ea9327 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:47.934421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-39d2932c-e4a4-4676-b24b-a168596b217e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:52:47.934421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:52:47.934421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:52:47.934421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:52:47.934421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:52:47.934421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:52:47.934421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:52:47.934421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:52:47.934421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:52:47.934421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:52:47.934421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:52:47.934421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:52:47.934421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:52:47.934421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:52:47.934421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:52:47.934421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:52:47.935252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:52:47.935252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:52:47.935252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:52:47.935252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:52:47.935252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:52:47.935252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:52:47.935252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:52:47.935252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:52:47.935252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:52:47.935252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:52:47.935252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:52:47.935252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:52:47.935252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:52:47.935252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:52:47.935252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:52:47.935252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:52:47.938327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:52:47.938327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:52:47.938327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:52:47.938327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:52:47.938327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:52:47.938327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:52:47.938327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:52:47.938327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:52:47.938327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:52:47.938327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:52:47.938327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:52:47.938327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:52:47.938327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 35/69] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:52:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:52:47.938327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f299d86-708c-4bfd-94d8-f4a2bdc6a9fe None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:47.940028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f299d86-708c-4bfd-94d8-f4a2bdc6a9fe None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:47.940028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f299d86-708c-4bfd-94d8-f4a2bdc6a9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:47.955727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3f299d86-708c-4bfd-94d8-f4a2bdc6a9fe None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:47.956946 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3f299d86-708c-4bfd-94d8-f4a2bdc6a9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:48.033482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 36/70] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:52:48.034991 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733f498-11f5-4ad4-911c-c848464b3164 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:48.035350 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733f498-11f5-4ad4-911c-c848464b3164 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:48.035685 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733f498-11f5-4ad4-911c-c848464b3164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:48.054972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b733f498-11f5-4ad4-911c-c848464b3164 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:48.056207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b733f498-11f5-4ad4-911c-c848464b3164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:48.059460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0747b-a836-4810-99cc-545e26ea9327 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:48.095210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f740a2f-16f6-4c86-8445-27dc1a1870c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:48.095882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f740a2f-16f6-4c86-8445-27dc1a1870c5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:48.096170 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f740a2f-16f6-4c86-8445-27dc1a1870c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:48.096452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f740a2f-16f6-4c86-8445-27dc1a1870c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:48.097226 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f740a2f-16f6-4c86-8445-27dc1a1870c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:48.097540 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f740a2f-16f6-4c86-8445-27dc1a1870c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:48.098953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f740a2f-16f6-4c86-8445-27dc1a1870c5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:48.107943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 35/71] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:52:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:48.108722 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f740a2f-16f6-4c86-8445-27dc1a1870c5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:48.124918 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 37/72] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:52:48.140605 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7ea2b-143a-430d-a47e-29b008447ec0 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:48.140605 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7ea2b-143a-430d-a47e-29b008447ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:48.140605 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7ea2b-143a-430d-a47e-29b008447ec0 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:48.140605 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 38/73] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:48.140605 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7ea2b-143a-430d-a47e-29b008447ec0 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:48.163536 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:48.166253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:48.219744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f740a2f-16f6-4c86-8445-27dc1a1870c5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:48.232223 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7ea2b-143a-430d-a47e-29b008447ec0 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:48.254781 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e0845-350a-474e-a348-8c947cc829cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:48.255458 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e0845-350a-474e-a348-8c947cc829cd None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:48.255794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e0845-350a-474e-a348-8c947cc829cd None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:48.256078 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e0845-350a-474e-a348-8c947cc829cd None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:48.256896 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e0845-350a-474e-a348-8c947cc829cd None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:48.257197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e0845-350a-474e-a348-8c947cc829cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:48.258585 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e0845-350a-474e-a348-8c947cc829cd None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:48.265455 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f7885-832c-4bb9-a62a-344d97d2408e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:48.266274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f7885-832c-4bb9-a62a-344d97d2408e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:48.266866 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f7885-832c-4bb9-a62a-344d97d2408e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:48.267140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f7885-832c-4bb9-a62a-344d97d2408e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:48.267465 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 36/74] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:52:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:48.268262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e0845-350a-474e-a348-8c947cc829cd None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:48.282875 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f7885-832c-4bb9-a62a-344d97d2408e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:52:48.291026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-842f7885-832c-4bb9-a62a-344d97d2408e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:52:48.291026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:52:48.291026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:52:48.291026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:52:48.291026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:52:48.291026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:52:48.291026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:52:48.291026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:52:48.291026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:52:48.291026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:52:48.291026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:52:48.291026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:52:48.291026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:52:48.291026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:52:48.291026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:52:48.291026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:52:48.292100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:52:48.292100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:52:48.292100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:52:48.292100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:52:48.292100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:52:48.292100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:52:48.292100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:52:48.292100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:52:48.292100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:52:48.292100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:52:48.292100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:52:48.292100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:52:48.292100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:52:48.292100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:52:48.292100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:52:48.292100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:52:48.292958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:52:48.292958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:52:48.292958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:52:48.292958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:52:48.292958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:52:48.292958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:52:48.292958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:52:48.292958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:52:48.292958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:52:48.292958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:52:48.292958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:52:48.292958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:52:48.292958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 39/75] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:52:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:52:48.293834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200edcba-070d-4750-a662-9241bafd4793 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:48.293897 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200edcba-070d-4750-a662-9241bafd4793 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:48.294191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200edcba-070d-4750-a662-9241bafd4793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:48.310547 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-200edcba-070d-4750-a662-9241bafd4793 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:48.311746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-200edcba-070d-4750-a662-9241bafd4793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:48.373911 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e0845-350a-474e-a348-8c947cc829cd None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:48.380446 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 40/76] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:52:48.381316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200edcba-070d-4750-a662-9241bafd4793 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:48.393033 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:48.409849 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:48.412852 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ba435-ad18-43dd-a28d-ece59016975b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:48.413874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ba435-ad18-43dd-a28d-ece59016975b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:48.414161 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ba435-ad18-43dd-a28d-ece59016975b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:48.414442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ba435-ad18-43dd-a28d-ece59016975b None admin] PATH_INFO: `/v3/users/c132d4e24cfd4bd197338f840d4d9eb9` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:48.415692 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ba435-ad18-43dd-a28d-ece59016975b None admin] RBAC: Authorizing `identity:get_user(user_id=c132d4e24cfd4bd197338f840d4d9eb9)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:48.417691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ba435-ad18-43dd-a28d-ece59016975b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:48.419362 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 37/77] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:52:48 2022] GET /identity/v3/users/c132d4e24cfd4bd197338f840d4d9eb9 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:48.420133 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ba435-ad18-43dd-a28d-ece59016975b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:48.481734 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200edcba-070d-4750-a662-9241bafd4793 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:48.515651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593be49-b83b-4ac0-a68c-b447c4e4f06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:48.516786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593be49-b83b-4ac0-a68c-b447c4e4f06b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:48.517069 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593be49-b83b-4ac0-a68c-b447c4e4f06b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:48.517351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593be49-b83b-4ac0-a68c-b447c4e4f06b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:48.521486 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ba435-ad18-43dd-a28d-ece59016975b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:48.527454 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5593be49-b83b-4ac0-a68c-b447c4e4f06b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:52:48.534984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-5593be49-b83b-4ac0-a68c-b447c4e4f06b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:52:48.534984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:52:48.534984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:52:48.534984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:52:48.534984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:52:48.534984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:52:48.534984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:52:48.534984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:52:48.534984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:52:48.534984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:52:48.534984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:52:48.534984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:52:48.534984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:52:48.534984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:52:48.534984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:52:48.534984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:52:48.536277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:52:48.536277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:52:48.536277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:52:48.536277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:52:48.536277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:52:48.536277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:52:48.536277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:52:48.536277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:52:48.536277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:52:48.536277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:52:48.536277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:52:48.536277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:52:48.536277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:52:48.536277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:52:48.536277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:52:48.536277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:52:48.537068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:52:48.537068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:52:48.537068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:52:48.537068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:52:48.537068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:52:48.537068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:52:48.537068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:52:48.537068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:52:48.537068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:52:48.537068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:52:48.537068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:52:48.537068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:52:48.537068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 41/78] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:52:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:52:48.537068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593be49-b83b-4ac0-a68c-b447c4e4f06b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:48.565149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24828a4-f7d2-4ef7-af88-766b72709c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:48.566006 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24828a4-f7d2-4ef7-af88-766b72709c47 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:48.566289 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24828a4-f7d2-4ef7-af88-766b72709c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:48.566566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24828a4-f7d2-4ef7-af88-766b72709c47 None admin] PATH_INFO: `/v3/users/0233db48571f40ee92089c7c9fc98814` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:48.580524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24828a4-f7d2-4ef7-af88-766b72709c47 None admin] RBAC: Authorizing `identity:get_user(user_id=0233db48571f40ee92089c7c9fc98814)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:48.582015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24828a4-f7d2-4ef7-af88-766b72709c47 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:48.583746 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 38/79] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:52:48 2022] GET /identity/v3/users/0233db48571f40ee92089c7c9fc98814 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:48.584621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24828a4-f7d2-4ef7-af88-766b72709c47 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:48.615656 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:48.625796 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:48.641317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593be49-b83b-4ac0-a68c-b447c4e4f06b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:48.675981 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd951365-cc97-434e-9152-6a6cd5a9cdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:48.676977 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd951365-cc97-434e-9152-6a6cd5a9cdcb None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:48.677446 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd951365-cc97-434e-9152-6a6cd5a9cdcb None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:48.677854 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd951365-cc97-434e-9152-6a6cd5a9cdcb None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:48.678965 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd951365-cc97-434e-9152-6a6cd5a9cdcb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:48.679451 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd951365-cc97-434e-9152-6a6cd5a9cdcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:48.681568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd951365-cc97-434e-9152-6a6cd5a9cdcb None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:48.692466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24828a4-f7d2-4ef7-af88-766b72709c47 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:48.697430 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 42/80] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:52:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:48.697430 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd951365-cc97-434e-9152-6a6cd5a9cdcb None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:48.728267 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912841b-4063-4ebc-9add-621d37114177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:48.729218 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912841b-4063-4ebc-9add-621d37114177 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:48.729639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912841b-4063-4ebc-9add-621d37114177 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:48.730181 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912841b-4063-4ebc-9add-621d37114177 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:48.731411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912841b-4063-4ebc-9add-621d37114177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:48.732923 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912841b-4063-4ebc-9add-621d37114177 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:48.734550 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 39/81] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:52:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:48.736519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912841b-4063-4ebc-9add-621d37114177 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:48.797622 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd951365-cc97-434e-9152-6a6cd5a9cdcb None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:48.833691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:48.836461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912841b-4063-4ebc-9add-621d37114177 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:48.840256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f59d4-05b5-4384-b1d1-be18eabf92d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:48.840256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f59d4-05b5-4384-b1d1-be18eabf92d0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:48.840256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f59d4-05b5-4384-b1d1-be18eabf92d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:48.840256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f59d4-05b5-4384-b1d1-be18eabf92d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:48.861061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:48.871956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e6580-2d50-4e49-8103-cc75a85c455b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:48.872655 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e6580-2d50-4e49-8103-cc75a85c455b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:48.872940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e6580-2d50-4e49-8103-cc75a85c455b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:48.873217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e6580-2d50-4e49-8103-cc75a85c455b None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:48.873974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e6580-2d50-4e49-8103-cc75a85c455b None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:48.874281 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e6580-2d50-4e49-8103-cc75a85c455b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:48.875699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e6580-2d50-4e49-8103-cc75a85c455b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:48.887945 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 40/82] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:52:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:48.887945 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e6580-2d50-4e49-8103-cc75a85c455b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:48.917811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f59d4-05b5-4384-b1d1-be18eabf92d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:48.919305 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f59d4-05b5-4384-b1d1-be18eabf92d0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:48.991135 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e6580-2d50-4e49-8103-cc75a85c455b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:49.021260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 43/83] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:52:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:52:49.022207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f59d4-05b5-4384-b1d1-be18eabf92d0 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:49.025806 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40aee8-00d0-406e-be64-27facd159e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:49.026601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40aee8-00d0-406e-be64-27facd159e25 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:49.027088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40aee8-00d0-406e-be64-27facd159e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:49.027549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40aee8-00d0-406e-be64-27facd159e25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:49.060730 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:49.073275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:49.100880 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40aee8-00d0-406e-be64-27facd159e25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:49.102360 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40aee8-00d0-406e-be64-27facd159e25 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:49.124725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f59d4-05b5-4384-b1d1-be18eabf92d0 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:49.162041 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b73ab-179e-477e-9791-43def416b6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:49.162841 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b73ab-179e-477e-9791-43def416b6fa None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:49.163343 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b73ab-179e-477e-9791-43def416b6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:49.163465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b73ab-179e-477e-9791-43def416b6fa None admin] PATH_INFO: `/v3/users/0233db48571f40ee92089c7c9fc98814` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:49.164885 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b73ab-179e-477e-9791-43def416b6fa None admin] RBAC: Authorizing `identity:get_user(user_id=0233db48571f40ee92089c7c9fc98814)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:49.166583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b73ab-179e-477e-9791-43def416b6fa None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:49.168241 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 44/84] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:52:49 2022] GET /identity/v3/users/0233db48571f40ee92089c7c9fc98814 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:49.168998 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b73ab-179e-477e-9791-43def416b6fa None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:49.202781 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 41/85] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:52:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:52:49.203630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40aee8-00d0-406e-be64-27facd159e25 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:49.262753 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:49.281091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b73ab-179e-477e-9791-43def416b6fa None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:49.297823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:49.304076 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40aee8-00d0-406e-be64-27facd159e25 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:49.316437 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c9e2a-5941-42f7-96b3-327863ff3fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:49.317118 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c9e2a-5941-42f7-96b3-327863ff3fa3 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:49.317399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c9e2a-5941-42f7-96b3-327863ff3fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:49.317644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c9e2a-5941-42f7-96b3-327863ff3fa3 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:49.318652 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c9e2a-5941-42f7-96b3-327863ff3fa3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:49.320578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c9e2a-5941-42f7-96b3-327863ff3fa3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:49.341416 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956350e-cc9d-4692-b4be-5c360f1c4e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:49.342302 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956350e-cc9d-4692-b4be-5c360f1c4e73 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:49.342582 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956350e-cc9d-4692-b4be-5c360f1c4e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:49.342862 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956350e-cc9d-4692-b4be-5c360f1c4e73 None admin] PATH_INFO: `/v3/users/0233db48571f40ee92089c7c9fc98814` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:49.344063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956350e-cc9d-4692-b4be-5c360f1c4e73 None admin] RBAC: Authorizing `identity:get_user(user_id=0233db48571f40ee92089c7c9fc98814)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:49.345588 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956350e-cc9d-4692-b4be-5c360f1c4e73 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:49.347024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 45/86] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Jun 25 15:52:49 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:52:49.347417 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 42/87] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:52:49 2022] GET /identity/v3/users/0233db48571f40ee92089c7c9fc98814 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:49.347860 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c9e2a-5941-42f7-96b3-327863ff3fa3 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:49.347955 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956350e-cc9d-4692-b4be-5c360f1c4e73 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:49.461908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956350e-cc9d-4692-b4be-5c360f1c4e73 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:49.468273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c9e2a-5941-42f7-96b3-327863ff3fa3 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:49.496446 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:49.503861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b53f6-9051-483e-af4c-824a110aa381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:49.504579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b53f6-9051-483e-af4c-824a110aa381 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:49.505447 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b53f6-9051-483e-af4c-824a110aa381 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:49.505447 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b53f6-9051-483e-af4c-824a110aa381 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:49.509080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5447b4-c068-4e0d-8bc9-a543eed0da36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:49.509989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5447b4-c068-4e0d-8bc9-a543eed0da36 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:49.510290 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5447b4-c068-4e0d-8bc9-a543eed0da36 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:49.510570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5447b4-c068-4e0d-8bc9-a543eed0da36 None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:49.511819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5447b4-c068-4e0d-8bc9-a543eed0da36 None admin] RBAC: Authorizing `identity:get_project(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:49.513343 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5447b4-c068-4e0d-8bc9-a543eed0da36 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:49.515072 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 46/88] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:52:49 2022] GET /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:49.516280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5447b4-c068-4e0d-8bc9-a543eed0da36 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:49.568731 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:49.605870 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b53f6-9051-483e-af4c-824a110aa381 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:49.607383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b53f6-9051-483e-af4c-824a110aa381 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:49.647187 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5447b4-c068-4e0d-8bc9-a543eed0da36 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:49.685361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b96c2c-b1ed-4750-ba22-221a400b405c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:49.686341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b96c2c-b1ed-4750-ba22-221a400b405c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:49.686632 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b96c2c-b1ed-4750-ba22-221a400b405c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:49.686916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b96c2c-b1ed-4750-ba22-221a400b405c None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:49.688157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b96c2c-b1ed-4750-ba22-221a400b405c None admin] RBAC: Authorizing `identity:get_project(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:49.689672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b96c2c-b1ed-4750-ba22-221a400b405c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:49.691418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 47/89] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:52:49 2022] GET /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:49.692204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b96c2c-b1ed-4750-ba22-221a400b405c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:49.735776 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 43/90] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:52:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:52:49.736592 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b53f6-9051-483e-af4c-824a110aa381 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:49.782833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:49.814760 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b96c2c-b1ed-4750-ba22-221a400b405c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:49.825545 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:49.855848 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df83c15-58fd-4469-b995-f6b923e2b040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:49.856793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df83c15-58fd-4469-b995-f6b923e2b040 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:49.857225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df83c15-58fd-4469-b995-f6b923e2b040 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:49.857645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df83c15-58fd-4469-b995-f6b923e2b040 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:49.881834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b53f6-9051-483e-af4c-824a110aa381 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:49.932073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044536b7-4133-4e93-bd28-b3e9e5603d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:49.932826 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044536b7-4133-4e93-bd28-b3e9e5603d02 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:49.933164 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044536b7-4133-4e93-bd28-b3e9e5603d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:49.933441 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044536b7-4133-4e93-bd28-b3e9e5603d02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:49.934588 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044536b7-4133-4e93-bd28-b3e9e5603d02 None admin] RBAC: Adding query filter params (role.id=b1e13c449fa64b3e88b6832bb767214b, scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea, user.id=c132d4e24cfd4bd197338f840d4d9eb9) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:49.934936 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044536b7-4133-4e93-bd28-b3e9e5603d02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:49.937574 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044536b7-4133-4e93-bd28-b3e9e5603d02 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:49.938084 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df83c15-58fd-4469-b995-f6b923e2b040 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:49.939712 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df83c15-58fd-4469-b995-f6b923e2b040 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:49.944757 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 44/91] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:52:49 2022] GET /identity/v3/role_assignments?user.id=c132d4e24cfd4bd197338f840d4d9eb9&scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:49.945674 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044536b7-4133-4e93-bd28-b3e9e5603d02 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:50.065727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 48/92] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:52:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:52:50.066857 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df83c15-58fd-4469-b995-f6b923e2b040 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:50.073686 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:50.079308 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:50.085707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044536b7-4133-4e93-bd28-b3e9e5603d02 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:50.132837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3443353-56b8-491d-a03c-78115b5240d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:50.133524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3443353-56b8-491d-a03c-78115b5240d7 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:50.133804 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3443353-56b8-491d-a03c-78115b5240d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:50.134080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3443353-56b8-491d-a03c-78115b5240d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:50.134960 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3443353-56b8-491d-a03c-78115b5240d7 None admin] RBAC: Adding query filter params (role.id=8b9bae8f81854757b4f389d3bd22f595, scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea, user.id=0233db48571f40ee92089c7c9fc98814) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:50.135306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3443353-56b8-491d-a03c-78115b5240d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:50.137015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3443353-56b8-491d-a03c-78115b5240d7 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:50.155847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 45/93] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:52:49 2022] GET /identity/v3/role_assignments?user.id=0233db48571f40ee92089c7c9fc98814&scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea&role.id=8b9bae8f81854757b4f389d3bd22f595 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:50.158931 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3443353-56b8-491d-a03c-78115b5240d7 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:50.260650 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df83c15-58fd-4469-b995-f6b923e2b040 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:50.315460 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d47f78-b99d-4280-b355-ee237c1ffe5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:50.316495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d47f78-b99d-4280-b355-ee237c1ffe5f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:50.316727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d47f78-b99d-4280-b355-ee237c1ffe5f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:50.317007 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d47f78-b99d-4280-b355-ee237c1ffe5f None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:50.320921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d47f78-b99d-4280-b355-ee237c1ffe5f None admin] RBAC: Authorizing `identity:get_project(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:50.322457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d47f78-b99d-4280-b355-ee237c1ffe5f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:50.328182 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 49/94] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:52:50 2022] GET /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:50.334598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d47f78-b99d-4280-b355-ee237c1ffe5f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:50.367743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3443353-56b8-491d-a03c-78115b5240d7 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:50.435449 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec8d46-28c7-4f24-9add-4e8b6f8b8a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:50.436336 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec8d46-28c7-4f24-9add-4e8b6f8b8a8f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:50.436645 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec8d46-28c7-4f24-9add-4e8b6f8b8a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:50.440177 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec8d46-28c7-4f24-9add-4e8b6f8b8a8f None admin] PATH_INFO: `/v3/projects/7fda594bccc94e469c094a50537682fa` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:50.441534 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec8d46-28c7-4f24-9add-4e8b6f8b8a8f None admin] RBAC: Authorizing `identity:get_project(project_id=7fda594bccc94e469c094a50537682fa)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:50.443018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec8d46-28c7-4f24-9add-4e8b6f8b8a8f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:50.445212 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 46/95] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:52:50 2022] GET /identity/v3/projects/7fda594bccc94e469c094a50537682fa => generated 312 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:50.450587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec8d46-28c7-4f24-9add-4e8b6f8b8a8f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:50.494461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:50.542901 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:50.605382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d47f78-b99d-4280-b355-ee237c1ffe5f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:50.634799 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec8d46-28c7-4f24-9add-4e8b6f8b8a8f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:50.676090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537b6e7-d67c-4bd3-a782-b20e84b3e840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:50.677273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537b6e7-d67c-4bd3-a782-b20e84b3e840 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:50.677793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537b6e7-d67c-4bd3-a782-b20e84b3e840 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:50.678271 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537b6e7-d67c-4bd3-a782-b20e84b3e840 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:50.679348 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537b6e7-d67c-4bd3-a782-b20e84b3e840 None admin] RBAC: Adding query filter params (role.id=6322fb4a438d4a389c43d05248665e72, scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea, user.id=0233db48571f40ee92089c7c9fc98814) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:50.679874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537b6e7-d67c-4bd3-a782-b20e84b3e840 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:50.682825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537b6e7-d67c-4bd3-a782-b20e84b3e840 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:50.698827 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 50/96] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:52:50 2022] GET /identity/v3/role_assignments?user.id=0233db48571f40ee92089c7c9fc98814&scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea&role.id=6322fb4a438d4a389c43d05248665e72 => generated 259 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:50.702569 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b593078-263e-41d0-9b73-f80127740db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:50.703502 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b593078-263e-41d0-9b73-f80127740db8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:50.704013 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b593078-263e-41d0-9b73-f80127740db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:50.704542 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b593078-263e-41d0-9b73-f80127740db8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:50.705684 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b593078-263e-41d0-9b73-f80127740db8 None admin] RBAC: Adding query filter params (role.id=8b9bae8f81854757b4f389d3bd22f595, scope.project.id=7fda594bccc94e469c094a50537682fa, user.id=0233db48571f40ee92089c7c9fc98814) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:50.706220 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b593078-263e-41d0-9b73-f80127740db8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:50.707954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b593078-263e-41d0-9b73-f80127740db8 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:50.715100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 47/97] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:52:50 2022] GET /identity/v3/role_assignments?user.id=0233db48571f40ee92089c7c9fc98814&scope.project.id=7fda594bccc94e469c094a50537682fa&role.id=8b9bae8f81854757b4f389d3bd22f595 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:51.956699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169c210-a2d1-46f7-b19b-d7928bc5e2ca None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:51.957207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169c210-a2d1-46f7-b19b-d7928bc5e2ca None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:51.957278 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169c210-a2d1-46f7-b19b-d7928bc5e2ca None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:51.958923 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 51/98] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:51.964627 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180262e-862d-40f3-a496-28876eb96194 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:51.965168 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180262e-862d-40f3-a496-28876eb96194 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:51.965580 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180262e-862d-40f3-a496-28876eb96194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:51.993473 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-8180262e-862d-40f3-a496-28876eb96194 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:51.994951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8180262e-862d-40f3-a496-28876eb96194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:52.076243 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 48/99] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:52:52.120466 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169c210-a2d1-46f7-b19b-d7928bc5e2ca None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:52.236471 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169c210-a2d1-46f7-b19b-d7928bc5e2ca None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:52.272436 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9336cb7-3fd3-4543-8da5-ed0de90b024c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:52.273265 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9336cb7-3fd3-4543-8da5-ed0de90b024c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:52.273552 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9336cb7-3fd3-4543-8da5-ed0de90b024c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:52.273832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9336cb7-3fd3-4543-8da5-ed0de90b024c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:52.284807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:52.292722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9336cb7-3fd3-4543-8da5-ed0de90b024c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:52:52.300654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-b9336cb7-3fd3-4543-8da5-ed0de90b024c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 25 15:52:52.300654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:52:52.300654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:52:52.300654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:52:52.300654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:52:52.300654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:52:52.300654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:52:52.300654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:52:52.300654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:52:52.300654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:52:52.300654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:52:52.300654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:52:52.300654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:52:52.300654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:52:52.300654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:52:52.300654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:52:52.301530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:52:52.301530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:52:52.301530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:52:52.301530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:52:52.301530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:52:52.301530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:52:52.301530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:52:52.301530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:52:52.301530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:52:52.301530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:52:52.301530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:52:52.301530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:52:52.301530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:52:52.301530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:52:52.301530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:52:52.301530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:52:52.302285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:52:52.302285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:52:52.302285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:52:52.302285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:52:52.302285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:52:52.302285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:52:52.302285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:52:52.302285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:52:52.302285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:52:52.302285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:52:52.302285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 25 15:52:52.302285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:52:52.302285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 52/100] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:52:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:52:52.309226 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8180262e-862d-40f3-a496-28876eb96194 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:52.332059 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:52.425340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8180262e-862d-40f3-a496-28876eb96194 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:52.467586 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4abcc28-9066-4ce7-89a0-b56b1d86f253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:52.468740 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abcc28-9066-4ce7-89a0-b56b1d86f253 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:52.469213 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abcc28-9066-4ce7-89a0-b56b1d86f253 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:52.469689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abcc28-9066-4ce7-89a0-b56b1d86f253 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:52.470636 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4abcc28-9066-4ce7-89a0-b56b1d86f253 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:52.471126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4abcc28-9066-4ce7-89a0-b56b1d86f253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:52.472754 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4abcc28-9066-4ce7-89a0-b56b1d86f253 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:52.482550 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 49/101] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:52:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:52.489535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9336cb7-3fd3-4543-8da5-ed0de90b024c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:52.606501 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9336cb7-3fd3-4543-8da5-ed0de90b024c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:52.649996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb1ac9-5027-4468-9753-65cac9b399bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:52.650973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb1ac9-5027-4468-9753-65cac9b399bd None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:52.651423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb1ac9-5027-4468-9753-65cac9b399bd None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:52.651714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb1ac9-5027-4468-9753-65cac9b399bd None admin] PATH_INFO: `/v3/users/31ac558b116c43a28ab5c1050a8b4c42` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:52.674468 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb1ac9-5027-4468-9753-65cac9b399bd None admin] RBAC: Authorizing `identity:get_user(user_id=31ac558b116c43a28ab5c1050a8b4c42)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:52.676019 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb1ac9-5027-4468-9753-65cac9b399bd None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:52.677767 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 53/102] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:52:52 2022] GET /identity/v3/users/31ac558b116c43a28ab5c1050a8b4c42 => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:52.683323 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4abcc28-9066-4ce7-89a0-b56b1d86f253 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:52.697229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7ec49-47c4-46a5-b5fe-b50975e74584 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:52.697543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7ec49-47c4-46a5-b5fe-b50975e74584 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:52.697832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7ec49-47c4-46a5-b5fe-b50975e74584 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:52.699222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 54/103] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:52.704593 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4e20a-f518-44fc-af93-d89b6237e57d None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:52.704890 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4e20a-f518-44fc-af93-d89b6237e57d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:52.705187 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4e20a-f518-44fc-af93-d89b6237e57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:52.738950 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-47d4e20a-f518-44fc-af93-d89b6237e57d None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:52.740166 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-47d4e20a-f518-44fc-af93-d89b6237e57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:52.877488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:52.888214 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:52.908659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 55/104] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:52:52.910135 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c99a35-972c-4a03-975c-2583ba7023f0 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:52.910432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c99a35-972c-4a03-975c-2583ba7023f0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:52.910717 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c99a35-972c-4a03-975c-2583ba7023f0 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:52.912297 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 56/105] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:52.918558 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d619a-3816-4eb0-a94b-6d7f1bbd6d8f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:52.918852 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d619a-3816-4eb0-a94b-6d7f1bbd6d8f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:52.919140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d619a-3816-4eb0-a94b-6d7f1bbd6d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:52.934776 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4abcc28-9066-4ce7-89a0-b56b1d86f253 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:52.960577 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-993d619a-3816-4eb0-a94b-6d7f1bbd6d8f None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:52.961800 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-993d619a-3816-4eb0-a94b-6d7f1bbd6d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:52.992013 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42e51e-aeab-42fe-9704-07a744be4d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:52.993521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42e51e-aeab-42fe-9704-07a744be4d9d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:52.993993 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42e51e-aeab-42fe-9704-07a744be4d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:52.994530 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42e51e-aeab-42fe-9704-07a744be4d9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:53.034132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 57/106] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:52:53.035390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8fd6f-a18f-4dea-8188-b4c89e5ab94f None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:53.035687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8fd6f-a18f-4dea-8188-b4c89e5ab94f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:53.035970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8fd6f-a18f-4dea-8188-b4c89e5ab94f None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:53.037311 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 58/107] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:53.041575 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8fd6f-a18f-4dea-8188-b4c89e5ab94f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:53.074912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42e51e-aeab-42fe-9704-07a744be4d9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:53.077217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42e51e-aeab-42fe-9704-07a744be4d9d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:53.147411 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8fd6f-a18f-4dea-8188-b4c89e5ab94f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:53.163573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:53.169173 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:53.186471 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 50/108] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:52:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:52:53.187977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88dcb1-ded0-47f7-859e-7137ad135481 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:53.188389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88dcb1-ded0-47f7-859e-7137ad135481 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:53.188795 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88dcb1-ded0-47f7-859e-7137ad135481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:53.189250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3bea2-e096-496e-ac02-7e31274aff79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:53.190229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3bea2-e096-496e-ac02-7e31274aff79 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:53.190690 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3bea2-e096-496e-ac02-7e31274aff79 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:53.191120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3bea2-e096-496e-ac02-7e31274aff79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:53.203058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3bea2-e096-496e-ac02-7e31274aff79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:52:53.206642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-fc88dcb1-ded0-47f7-859e-7137ad135481 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:53.208008 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-fc88dcb1-ded0-47f7-859e-7137ad135481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:53.211637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-69f3bea2-e096-496e-ac02-7e31274aff79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:52:53.211637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:52:53.211637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:52:53.211637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:52:53.211637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:52:53.211637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:52:53.211637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:52:53.211637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:52:53.211637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:52:53.211637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:52:53.211637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:52:53.211637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:52:53.211637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:52:53.211637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:52:53.211637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:52:53.211637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:52:53.212709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:52:53.212709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:52:53.212709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:52:53.212709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:52:53.212709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:52:53.212709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:52:53.212709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:52:53.212709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:52:53.212709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:52:53.212709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:52:53.212709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:52:53.212709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:52:53.212709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:52:53.212709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:52:53.212709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:52:53.212709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:52:53.213579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:52:53.213579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:52:53.213579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:52:53.213579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:52:53.213579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:52:53.213579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:52:53.213579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:52:53.213579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:52:53.213579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:52:53.213579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:52:53.213579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:52:53.213579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:52:53.215870 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 59/109] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:52:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:52:53.221897 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3bea2-e096-496e-ac02-7e31274aff79 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:53.283507 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 51/110] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:52:53.289449 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88dcb1-ded0-47f7-859e-7137ad135481 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:53.350853 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3bea2-e096-496e-ac02-7e31274aff79 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:53.405727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c9aaf-ff61-42b2-8218-91f1cbd3deba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:53.407184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c9aaf-ff61-42b2-8218-91f1cbd3deba None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:53.407946 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c9aaf-ff61-42b2-8218-91f1cbd3deba None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:53.408434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c9aaf-ff61-42b2-8218-91f1cbd3deba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:53.410661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c9aaf-ff61-42b2-8218-91f1cbd3deba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:53.413346 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c9aaf-ff61-42b2-8218-91f1cbd3deba None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:53.416208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 60/111] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:52:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:53.417252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c9aaf-ff61-42b2-8218-91f1cbd3deba None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:53.418594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88dcb1-ded0-47f7-859e-7137ad135481 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:53.435582 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:53.466405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d205ac-f024-4deb-acb3-091dee7560e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:53.467222 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d205ac-f024-4deb-acb3-091dee7560e4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:53.467505 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d205ac-f024-4deb-acb3-091dee7560e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:53.467787 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d205ac-f024-4deb-acb3-091dee7560e4 None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:53.471044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d205ac-f024-4deb-acb3-091dee7560e4 None admin] RBAC: Authorizing `identity:get_project(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:53.472588 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d205ac-f024-4deb-acb3-091dee7560e4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:53.474564 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 52/112] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:52:53 2022] GET /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:53.476374 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d205ac-f024-4deb-acb3-091dee7560e4 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:53.489222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:53.554536 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c9aaf-ff61-42b2-8218-91f1cbd3deba None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:53.592471 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98885b-a66c-4e0d-80ee-9c985a366601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:53.593318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98885b-a66c-4e0d-80ee-9c985a366601 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:53.593675 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98885b-a66c-4e0d-80ee-9c985a366601 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:53.594019 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98885b-a66c-4e0d-80ee-9c985a366601 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:53.595079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98885b-a66c-4e0d-80ee-9c985a366601 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:53.595197 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98885b-a66c-4e0d-80ee-9c985a366601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:53.597301 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98885b-a66c-4e0d-80ee-9c985a366601 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:53.613655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 61/113] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:52:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:53.613655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98885b-a66c-4e0d-80ee-9c985a366601 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:53.636499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d205ac-f024-4deb-acb3-091dee7560e4 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:53.685754 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298f281-e7a5-4916-a619-b1e91966f472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:53.687633 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298f281-e7a5-4916-a619-b1e91966f472 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:53.688129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298f281-e7a5-4916-a619-b1e91966f472 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:53.688640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298f281-e7a5-4916-a619-b1e91966f472 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:53.705188 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298f281-e7a5-4916-a619-b1e91966f472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:52:53.716024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-d298f281-e7a5-4916-a619-b1e91966f472 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:52:53.716024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:52:53.716024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:52:53.716024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:52:53.716024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:52:53.716024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:52:53.716024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:52:53.716024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:52:53.716024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:52:53.716024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:52:53.716024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:52:53.716024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:52:53.716024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:52:53.716024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:52:53.716024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:52:53.716024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:52:53.716994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:52:53.716994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:52:53.716994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:52:53.716994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:52:53.716994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:52:53.716994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:52:53.716994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:52:53.716994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:52:53.716994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:52:53.716994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:52:53.716994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:52:53.716994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:52:53.716994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:52:53.716994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:52:53.716994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:52:53.716994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:52:53.717818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:52:53.717818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:52:53.717818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:52:53.717818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:52:53.717818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:52:53.717818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:52:53.717818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:52:53.717818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:52:53.717818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:52:53.717818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:52:53.717818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:52:53.717818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:52:53.720824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 53/114] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:52:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:52:53.721797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298f281-e7a5-4916-a619-b1e91966f472 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:53.737383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98885b-a66c-4e0d-80ee-9c985a366601 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:53.749672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:53.787611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:53.789073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ed60f-eb80-4411-a384-792254801fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:53.790015 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ed60f-eb80-4411-a384-792254801fe6 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:53.790473 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ed60f-eb80-4411-a384-792254801fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:53.791823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ed60f-eb80-4411-a384-792254801fe6 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:53.795146 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ed60f-eb80-4411-a384-792254801fe6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:53.797178 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ed60f-eb80-4411-a384-792254801fe6 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:53.848736 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 62/115] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Jun 25 15:52:53 2022] POST /identity/v3/projects => generated 302 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:52:53.850552 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ed60f-eb80-4411-a384-792254801fe6 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:53.920066 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298f281-e7a5-4916-a619-b1e91966f472 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:53.985430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f9d88-f5c4-427a-917f-eb7d41783f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:53.986044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f9d88-f5c4-427a-917f-eb7d41783f06 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:53.986329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f9d88-f5c4-427a-917f-eb7d41783f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:53.986609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f9d88-f5c4-427a-917f-eb7d41783f06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:53.987494 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f9d88-f5c4-427a-917f-eb7d41783f06 None admin] RBAC: Adding query filter params (role.id=f70856a334f7479fb00bac40f893e9f0, scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea, user.id=31ac558b116c43a28ab5c1050a8b4c42) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:53.987895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f9d88-f5c4-427a-917f-eb7d41783f06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:53.989440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f9d88-f5c4-427a-917f-eb7d41783f06 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:54.003476 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 54/116] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:52:53 2022] GET /identity/v3/role_assignments?user.id=31ac558b116c43a28ab5c1050a8b4c42&scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea&role.id=f70856a334f7479fb00bac40f893e9f0 => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:54.003476 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f9d88-f5c4-427a-917f-eb7d41783f06 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:54.034047 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ed60f-eb80-4411-a384-792254801fe6 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:54.074981 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42313eb1-92d0-433b-b99c-d70845f8ff65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:54.076462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42313eb1-92d0-433b-b99c-d70845f8ff65 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:54.077085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42313eb1-92d0-433b-b99c-d70845f8ff65 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:54.077587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42313eb1-92d0-433b-b99c-d70845f8ff65 None admin] PATH_INFO: `/v3/users/c132d4e24cfd4bd197338f840d4d9eb9` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:54.080220 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42313eb1-92d0-433b-b99c-d70845f8ff65 None admin] RBAC: Authorizing `identity:get_user(user_id=c132d4e24cfd4bd197338f840d4d9eb9)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:54.082190 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42313eb1-92d0-433b-b99c-d70845f8ff65 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:54.084512 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 63/117] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:52:53 2022] GET /identity/v3/users/c132d4e24cfd4bd197338f840d4d9eb9 => generated 247 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:54.086347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1ce83-ce1c-4dde-bf0a-1818090459e5 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:54.086916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1ce83-ce1c-4dde-bf0a-1818090459e5 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:54.087452 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1ce83-ce1c-4dde-bf0a-1818090459e5 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:54.089387 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 64/118] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:54.090962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a577fef-c4a1-41df-a3b6-a5e0fc024d50 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:54.091516 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a577fef-c4a1-41df-a3b6-a5e0fc024d50 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:54.092014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a577fef-c4a1-41df-a3b6-a5e0fc024d50 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:54.093651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 65/119] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:54.095185 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a577fef-c4a1-41df-a3b6-a5e0fc024d50 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:54.130191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:54.147197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f9d88-f5c4-427a-917f-eb7d41783f06 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:54.150115 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:54.230207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a577fef-c4a1-41df-a3b6-a5e0fc024d50 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:54.243052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fef7b-f740-47f4-9be4-c920918a2637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:54.243052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fef7b-f740-47f4-9be4-c920918a2637 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:54.243052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fef7b-f740-47f4-9be4-c920918a2637 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:54.243052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fef7b-f740-47f4-9be4-c920918a2637 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:54.245887 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104fef7b-f740-47f4-9be4-c920918a2637 None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:54.245887 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104fef7b-f740-47f4-9be4-c920918a2637 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:54.245887 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104fef7b-f740-47f4-9be4-c920918a2637 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:54.260531 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 55/120] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:52:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:54.262150 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a69c8a-a4f5-4c6b-9f44-371b64f18168 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:54.262346 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a69c8a-a4f5-4c6b-9f44-371b64f18168 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:54.263128 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a69c8a-a4f5-4c6b-9f44-371b64f18168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:54.303731 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e2a69c8a-a4f5-4c6b-9f44-371b64f18168 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:54.305514 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e2a69c8a-a4f5-4c6b-9f44-371b64f18168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:54.321484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6530ea7-b538-4571-a20e-209376ca46e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:54.321484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6530ea7-b538-4571-a20e-209376ca46e1 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:54.321484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6530ea7-b538-4571-a20e-209376ca46e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:54.321484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6530ea7-b538-4571-a20e-209376ca46e1 None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:54.324428 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6530ea7-b538-4571-a20e-209376ca46e1 None admin] RBAC: Authorizing `identity:get_project(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:54.324428 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6530ea7-b538-4571-a20e-209376ca46e1 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:54.326259 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 66/121] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:52:54 2022] GET /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:54.327499 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d40f7-b5bd-443a-ab73-d39921d1be60 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:54.329404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d40f7-b5bd-443a-ab73-d39921d1be60 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:54.329404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d40f7-b5bd-443a-ab73-d39921d1be60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:54.359053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-dc4d40f7-b5bd-443a-ab73-d39921d1be60 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:54.360668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-dc4d40f7-b5bd-443a-ab73-d39921d1be60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:54.412611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 56/122] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:52:54.418056 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a69c8a-a4f5-4c6b-9f44-371b64f18168 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:54.457020 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 67/123] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:52:54.457869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d40f7-b5bd-443a-ab73-d39921d1be60 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:54.493857 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:54.551272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:54.561777 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a69c8a-a4f5-4c6b-9f44-371b64f18168 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:54.582511 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d40f7-b5bd-443a-ab73-d39921d1be60 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:54.606985 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d291a48-52c9-42a0-b977-dcd681dbaa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:54.608116 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d291a48-52c9-42a0-b977-dcd681dbaa82 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:54.608819 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d291a48-52c9-42a0-b977-dcd681dbaa82 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:54.609106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d291a48-52c9-42a0-b977-dcd681dbaa82 None admin] PATH_INFO: `/v3/users/0233db48571f40ee92089c7c9fc98814` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:54.610850 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d291a48-52c9-42a0-b977-dcd681dbaa82 None admin] RBAC: Authorizing `identity:get_user(user_id=0233db48571f40ee92089c7c9fc98814)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:54.613050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d291a48-52c9-42a0-b977-dcd681dbaa82 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:54.615950 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 57/124] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:52:54 2022] GET /identity/v3/users/0233db48571f40ee92089c7c9fc98814 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:54.616722 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d291a48-52c9-42a0-b977-dcd681dbaa82 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:54.619861 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba181e-e75d-44ae-a830-90b9f0d91840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:54.620802 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba181e-e75d-44ae-a830-90b9f0d91840 None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:54.621086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba181e-e75d-44ae-a830-90b9f0d91840 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:54.621362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba181e-e75d-44ae-a830-90b9f0d91840 None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea/users/c132d4e24cfd4bd197338f840d4d9eb9/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:54.623370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba181e-e75d-44ae-a830-90b9f0d91840 None admin] RBAC: Authorizing `identity:create_grant(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea, user_id=c132d4e24cfd4bd197338f840d4d9eb9, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:54.625050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba181e-e75d-44ae-a830-90b9f0d91840 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:54.642751 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 68/125] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:52:54 2022] PUT /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea/users/c132d4e24cfd4bd197338f840d4d9eb9/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:52:54.643969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba181e-e75d-44ae-a830-90b9f0d91840 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:54.733197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d291a48-52c9-42a0-b977-dcd681dbaa82 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:54.772449 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:54.772633 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb42f5c-c17d-4f9a-a62e-bc7450c1528b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:54.782975 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb42f5c-c17d-4f9a-a62e-bc7450c1528b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:54.783293 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb42f5c-c17d-4f9a-a62e-bc7450c1528b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:54.783537 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb42f5c-c17d-4f9a-a62e-bc7450c1528b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:54.791635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba181e-e75d-44ae-a830-90b9f0d91840 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:54.817446 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb42f5c-c17d-4f9a-a62e-bc7450c1528b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:52:54.828764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-eeb42f5c-c17d-4f9a-a62e-bc7450c1528b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:52:54.828764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:52:54.828764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:52:54.828764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:52:54.828764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:52:54.828764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:52:54.828764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:52:54.828764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:52:54.828764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:52:54.828764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:52:54.828764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:52:54.828764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:52:54.828764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:52:54.828764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:52:54.828764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:52:54.828764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:52:54.829706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:52:54.829706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:52:54.829706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:52:54.829706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:52:54.829706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:52:54.829706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:52:54.829706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:52:54.829706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:52:54.829706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:52:54.829706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:52:54.829706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:52:54.829706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:52:54.829706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:52:54.829706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:52:54.829706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:52:54.829706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:52:54.830479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:52:54.830479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:52:54.830479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:52:54.830479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:52:54.830479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:52:54.830479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:52:54.830479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:52:54.830479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:52:54.830479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:52:54.830479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:52:54.830479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:52:54.830479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:52:54.832422 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 58/126] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:52:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:52:54.832422 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb42f5c-c17d-4f9a-a62e-bc7450c1528b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:54.839738 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3becb6ce-ff69-4fb5-9e82-df93cc17f689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:54.839738 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3becb6ce-ff69-4fb5-9e82-df93cc17f689 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:54.839738 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3becb6ce-ff69-4fb5-9e82-df93cc17f689 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:54.839738 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3becb6ce-ff69-4fb5-9e82-df93cc17f689 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:54.867103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3becb6ce-ff69-4fb5-9e82-df93cc17f689 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:52:54.880341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-3becb6ce-ff69-4fb5-9e82-df93cc17f689 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:52:54.880341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:52:54.880341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:52:54.880341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:52:54.880341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:52:54.880341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:52:54.880341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:52:54.880341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:52:54.880341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:52:54.880341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:52:54.880341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:52:54.880341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:52:54.880341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:52:54.880341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:52:54.880341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:52:54.880341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:52:54.881238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:52:54.881238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:52:54.881238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:52:54.881238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:52:54.881238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:52:54.881238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:52:54.881238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:52:54.881238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:52:54.881238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:52:54.881238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:52:54.881238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:52:54.881238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:52:54.881238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:52:54.881238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:52:54.881238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:52:54.881238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:52:54.882439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:52:54.882439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:52:54.882439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:52:54.882439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:52:54.882439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:52:54.882439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:52:54.882439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:52:54.882439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:52:54.882439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:52:54.882439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:52:54.882439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:52:54.882439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:52:54.882439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 69/127] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:52:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:52:54.882439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3becb6ce-ff69-4fb5-9e82-df93cc17f689 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:54.898641 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:55.000972 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb42f5c-c17d-4f9a-a62e-bc7450c1528b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:55.028264 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3becb6ce-ff69-4fb5-9e82-df93cc17f689 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:55.037452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342f9de-dc82-447a-81af-8035c54e358e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:55.038933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342f9de-dc82-447a-81af-8035c54e358e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:55.039416 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342f9de-dc82-447a-81af-8035c54e358e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:55.039883 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342f9de-dc82-447a-81af-8035c54e358e None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:55.041343 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342f9de-dc82-447a-81af-8035c54e358e None admin] RBAC: Authorizing `identity:get_project(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:55.043159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342f9de-dc82-447a-81af-8035c54e358e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:55.045168 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 59/128] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:52:54 2022] GET /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:55.050583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342f9de-dc82-447a-81af-8035c54e358e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:55.110124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4408424-e61b-4a25-b3e6-dce9a7cb93f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:55.110791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4408424-e61b-4a25-b3e6-dce9a7cb93f6 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:55.111086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4408424-e61b-4a25-b3e6-dce9a7cb93f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:55.111371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4408424-e61b-4a25-b3e6-dce9a7cb93f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:55.112132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4408424-e61b-4a25-b3e6-dce9a7cb93f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:55.112463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4408424-e61b-4a25-b3e6-dce9a7cb93f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:55.113882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4408424-e61b-4a25-b3e6-dce9a7cb93f6 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:55.134417 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 70/129] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:52:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:55.135343 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4408424-e61b-4a25-b3e6-dce9a7cb93f6 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:55.156794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:55.160939 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342f9de-dc82-447a-81af-8035c54e358e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:55.164273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:55.198299 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1338830-6e3d-4c76-af1c-dab760e74869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:55.199352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1338830-6e3d-4c76-af1c-dab760e74869 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:55.199828 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1338830-6e3d-4c76-af1c-dab760e74869 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:55.200229 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1338830-6e3d-4c76-af1c-dab760e74869 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:55.201262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1338830-6e3d-4c76-af1c-dab760e74869 None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:55.201723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1338830-6e3d-4c76-af1c-dab760e74869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:55.203258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1338830-6e3d-4c76-af1c-dab760e74869 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:55.212261 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 60/130] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:52:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:55.213538 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1338830-6e3d-4c76-af1c-dab760e74869 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:55.315990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1338830-6e3d-4c76-af1c-dab760e74869 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:55.355839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4408424-e61b-4a25-b3e6-dce9a7cb93f6 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:55.374458 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5831e2d-e110-4bef-95ee-1896a6defcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:55.375707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5831e2d-e110-4bef-95ee-1896a6defcb0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:55.375988 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5831e2d-e110-4bef-95ee-1896a6defcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:55.376257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5831e2d-e110-4bef-95ee-1896a6defcb0 None admin] PATH_INFO: `/v3/users/0233db48571f40ee92089c7c9fc98814` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:55.377557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5831e2d-e110-4bef-95ee-1896a6defcb0 None admin] RBAC: Authorizing `identity:get_user(user_id=0233db48571f40ee92089c7c9fc98814)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:55.380312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5831e2d-e110-4bef-95ee-1896a6defcb0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:55.382210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 61/131] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:52:55 2022] GET /identity/v3/users/0233db48571f40ee92089c7c9fc98814 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:55.382759 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5831e2d-e110-4bef-95ee-1896a6defcb0 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:55.427127 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a884f-9d3f-44e3-93f8-3efdb13f719a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:55.428027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a884f-9d3f-44e3-93f8-3efdb13f719a None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:55.428308 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a884f-9d3f-44e3-93f8-3efdb13f719a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:55.428617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a884f-9d3f-44e3-93f8-3efdb13f719a None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea/users/0233db48571f40ee92089c7c9fc98814/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:55.434386 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a884f-9d3f-44e3-93f8-3efdb13f719a None admin] RBAC: Authorizing `identity:create_grant(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea, user_id=0233db48571f40ee92089c7c9fc98814, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:55.435867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a884f-9d3f-44e3-93f8-3efdb13f719a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:55.439497 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:55.464633 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 71/132] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:52:55 2022] PUT /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea/users/0233db48571f40ee92089c7c9fc98814/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:52:55.465528 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a884f-9d3f-44e3-93f8-3efdb13f719a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:55.557735 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5831e2d-e110-4bef-95ee-1896a6defcb0 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:55.589638 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:55.603497 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115a6f8-ab95-4646-a249-ed13ae1b0eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:55.605052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115a6f8-ab95-4646-a249-ed13ae1b0eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:55.605489 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115a6f8-ab95-4646-a249-ed13ae1b0eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:55.605898 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115a6f8-ab95-4646-a249-ed13ae1b0eb5 None admin] PATH_INFO: `/v3/users/0233db48571f40ee92089c7c9fc98814` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:55.607308 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115a6f8-ab95-4646-a249-ed13ae1b0eb5 None admin] RBAC: Authorizing `identity:get_user(user_id=0233db48571f40ee92089c7c9fc98814)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:55.609762 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115a6f8-ab95-4646-a249-ed13ae1b0eb5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:55.612398 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 62/133] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:52:55 2022] GET /identity/v3/users/0233db48571f40ee92089c7c9fc98814 => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:55.618539 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115a6f8-ab95-4646-a249-ed13ae1b0eb5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:55.674645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a884f-9d3f-44e3-93f8-3efdb13f719a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:55.772081 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f1f8f-3ceb-4375-8d69-27c09466a7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:55.774009 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:55.774360 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f1f8f-3ceb-4375-8d69-27c09466a7ef None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:55.774645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f1f8f-3ceb-4375-8d69-27c09466a7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:55.774969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f1f8f-3ceb-4375-8d69-27c09466a7ef None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:55.778335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f1f8f-3ceb-4375-8d69-27c09466a7ef None admin] RBAC: Authorizing `identity:get_project(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:55.779821 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f1f8f-3ceb-4375-8d69-27c09466a7ef None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:55.781879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 72/134] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:52:55 2022] GET /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea => generated 298 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:55.787900 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f1f8f-3ceb-4375-8d69-27c09466a7ef None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:55.794512 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115a6f8-ab95-4646-a249-ed13ae1b0eb5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:55.862016 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c5661-f9ee-4781-9a5c-6bb4dcdc66a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:55.862900 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c5661-f9ee-4781-9a5c-6bb4dcdc66a8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:55.863085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c5661-f9ee-4781-9a5c-6bb4dcdc66a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:55.863359 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c5661-f9ee-4781-9a5c-6bb4dcdc66a8 None admin] PATH_INFO: `/v3/projects/7fda594bccc94e469c094a50537682fa` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:55.866477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c5661-f9ee-4781-9a5c-6bb4dcdc66a8 None admin] RBAC: Authorizing `identity:get_project(project_id=7fda594bccc94e469c094a50537682fa)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:55.867955 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c5661-f9ee-4781-9a5c-6bb4dcdc66a8 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:55.869838 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 63/135] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:52:55 2022] GET /identity/v3/projects/7fda594bccc94e469c094a50537682fa => generated 312 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:52:55.876313 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c5661-f9ee-4781-9a5c-6bb4dcdc66a8 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:55.990343 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f1f8f-3ceb-4375-8d69-27c09466a7ef None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:56.049530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7b718-4e99-45aa-aa75-302839aa53c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:56.050480 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7b718-4e99-45aa-aa75-302839aa53c9 None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:56.050761 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7b718-4e99-45aa-aa75-302839aa53c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:56.051035 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7b718-4e99-45aa-aa75-302839aa53c9 None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea/users/0233db48571f40ee92089c7c9fc98814/roles/6322fb4a438d4a389c43d05248665e72` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:56.054690 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7b718-4e99-45aa-aa75-302839aa53c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea, user_id=0233db48571f40ee92089c7c9fc98814, role_id=6322fb4a438d4a389c43d05248665e72)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:56.056149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7b718-4e99-45aa-aa75-302839aa53c9 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:56.062710 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:56.067812 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c5661-f9ee-4781-9a5c-6bb4dcdc66a8 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:56.081358 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 73/136] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:52:55 2022] PUT /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea/users/0233db48571f40ee92089c7c9fc98814/roles/6322fb4a438d4a389c43d05248665e72 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:52:56.142193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5965b2-7448-41de-9fb0-1917ffdc04a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:56.143050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5965b2-7448-41de-9fb0-1917ffdc04a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:56.143329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5965b2-7448-41de-9fb0-1917ffdc04a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:56.143601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5965b2-7448-41de-9fb0-1917ffdc04a9 None admin] PATH_INFO: `/v3/projects/7fda594bccc94e469c094a50537682fa/users/0233db48571f40ee92089c7c9fc98814/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:56.149995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5965b2-7448-41de-9fb0-1917ffdc04a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fda594bccc94e469c094a50537682fa, user_id=0233db48571f40ee92089c7c9fc98814, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:56.151534 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5965b2-7448-41de-9fb0-1917ffdc04a9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:56.174254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 64/137] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:52:55 2022] PUT /identity/v3/projects/7fda594bccc94e469c094a50537682fa/users/0233db48571f40ee92089c7c9fc98814/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 15:52:56.215519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeaece6-10f7-411a-a52b-6b3c235e39e2 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:56.216061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeaece6-10f7-411a-a52b-6b3c235e39e2 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:56.216061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeaece6-10f7-411a-a52b-6b3c235e39e2 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:56.217947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 74/138] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:56.223375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1c14b-db5c-443c-ab87-46dd3fb550e5 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:56.223657 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1c14b-db5c-443c-ab87-46dd3fb550e5 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:56.223938 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1c14b-db5c-443c-ab87-46dd3fb550e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:56.244894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-01b1c14b-db5c-443c-ab87-46dd3fb550e5 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:56.246164 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-01b1c14b-db5c-443c-ab87-46dd3fb550e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:56.264682 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:56.333329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 65/139] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:52:56.369789 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeaece6-10f7-411a-a52b-6b3c235e39e2 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:56.487827 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeaece6-10f7-411a-a52b-6b3c235e39e2 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:56.523576 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1e8de-e262-43c6-b156-3b44321a70d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:56.524748 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1e8de-e262-43c6-b156-3b44321a70d0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:56.525220 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1e8de-e262-43c6-b156-3b44321a70d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:56.525802 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1e8de-e262-43c6-b156-3b44321a70d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:56.527232 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1e8de-e262-43c6-b156-3b44321a70d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:56.528941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1e8de-e262-43c6-b156-3b44321a70d0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:56.531138 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 75/140] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:52:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:56.537319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1c14b-db5c-443c-ab87-46dd3fb550e5 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:56.672578 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1c14b-db5c-443c-ab87-46dd3fb550e5 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:56.718998 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4d4fb-55ff-4f4a-8d00-44a1cd3497ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:56.720053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4d4fb-55ff-4f4a-8d00-44a1cd3497ec None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:56.720563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4d4fb-55ff-4f4a-8d00-44a1cd3497ec None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:56.721044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4d4fb-55ff-4f4a-8d00-44a1cd3497ec None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:56.721968 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4d4fb-55ff-4f4a-8d00-44a1cd3497ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:56.723604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4d4fb-55ff-4f4a-8d00-44a1cd3497ec None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:56.734117 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa913e-61e1-439d-abf7-3e50b9748588 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:56.734773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa913e-61e1-439d-abf7-3e50b9748588 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:56.735227 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa913e-61e1-439d-abf7-3e50b9748588 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:56.737390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 76/141] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:56.742622 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bff69f-4f4d-48a9-b106-6f00b64ab072 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:56.742899 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bff69f-4f4d-48a9-b106-6f00b64ab072 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:56.743181 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bff69f-4f4d-48a9-b106-6f00b64ab072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:56.761637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 66/142] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Jun 25 15:52:56 2022] POST /identity/v3/users => generated 283 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:52:56.767427 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-84bff69f-4f4d-48a9-b106-6f00b64ab072 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:56.768771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-84bff69f-4f4d-48a9-b106-6f00b64ab072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:56.874098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:56.885075 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 77/143] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:52:56.923315 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4d4fb-55ff-4f4a-8d00-44a1cd3497ec None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:56.927517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:57.057238 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4d4fb-55ff-4f4a-8d00-44a1cd3497ec None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:57.096840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15add5a8-5eff-46b8-b0ed-276faed28924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:57.098437 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15add5a8-5eff-46b8-b0ed-276faed28924 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:57.098917 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15add5a8-5eff-46b8-b0ed-276faed28924 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:57.099408 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15add5a8-5eff-46b8-b0ed-276faed28924 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:57.113717 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15add5a8-5eff-46b8-b0ed-276faed28924 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:52:57.121953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-15add5a8-5eff-46b8-b0ed-276faed28924 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 25 15:52:57.121953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:52:57.121953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:52:57.121953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:52:57.121953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:52:57.121953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:52:57.121953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:52:57.121953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:52:57.121953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:52:57.121953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:52:57.121953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:52:57.121953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:52:57.121953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:52:57.121953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:52:57.121953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:52:57.121953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:52:57.122816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:52:57.122816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:52:57.122816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:52:57.122816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:52:57.122816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:52:57.122816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:52:57.122816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:52:57.122816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:52:57.122816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:52:57.122816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:52:57.122816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:52:57.122816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:52:57.122816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:52:57.122816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:52:57.122816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:52:57.122816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:52:57.123552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:52:57.123552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:52:57.123552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:52:57.123552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:52:57.123552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:52:57.123552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:52:57.123552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:52:57.123552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:52:57.123552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:52:57.123552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:52:57.123552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 25 15:52:57.123552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:52:57.125948 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 67/144] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:52:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:52:57.132041 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bff69f-4f4d-48a9-b106-6f00b64ab072 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:57.282287 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bff69f-4f4d-48a9-b106-6f00b64ab072 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:57.334134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb57d5-be6c-485b-a455-e755544f9419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:57.334742 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb57d5-be6c-485b-a455-e755544f9419 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:57.335028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb57d5-be6c-485b-a455-e755544f9419 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:57.335319 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb57d5-be6c-485b-a455-e755544f9419 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:57.336092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb57d5-be6c-485b-a455-e755544f9419 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:57.336400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb57d5-be6c-485b-a455-e755544f9419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:57.337829 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb57d5-be6c-485b-a455-e755544f9419 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:57.348026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 78/145] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:52:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:57.353436 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15add5a8-5eff-46b8-b0ed-276faed28924 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:57.423792 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:57.479378 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15add5a8-5eff-46b8-b0ed-276faed28924 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:57.524006 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5d576-6168-44d7-9a9e-63f66ae7d652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:57.525279 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5d576-6168-44d7-9a9e-63f66ae7d652 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:57.525826 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5d576-6168-44d7-9a9e-63f66ae7d652 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:57.526282 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5d576-6168-44d7-9a9e-63f66ae7d652 None admin] PATH_INFO: `/v3/users/31ac558b116c43a28ab5c1050a8b4c42` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:57.527738 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5d576-6168-44d7-9a9e-63f66ae7d652 None admin] RBAC: Authorizing `identity:get_user(user_id=31ac558b116c43a28ab5c1050a8b4c42)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:57.529421 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5d576-6168-44d7-9a9e-63f66ae7d652 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:57.531269 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 68/146] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:52:57 2022] GET /identity/v3/users/31ac558b116c43a28ab5c1050a8b4c42 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:57.536807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb57d5-be6c-485b-a455-e755544f9419 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:57.578469 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4df0e6-558c-44e0-912e-673257f098cb None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:57.578780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4df0e6-558c-44e0-912e-673257f098cb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:57.579032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4df0e6-558c-44e0-912e-673257f098cb None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:57.580460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 69/147] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:52:57.585707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15da809-5759-4ef2-a769-6945f2f2bf6c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:57.586039 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15da809-5759-4ef2-a769-6945f2f2bf6c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:57.586410 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15da809-5759-4ef2-a769-6945f2f2bf6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:57.607534 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e15da809-5759-4ef2-a769-6945f2f2bf6c None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:57.608750 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:57.609177 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e15da809-5759-4ef2-a769-6945f2f2bf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:57.666548 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb57d5-be6c-485b-a455-e755544f9419 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:57.701407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 70/148] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:52:57.714339 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39840b-8eca-43ce-9833-9be42ff16881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:57.715334 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39840b-8eca-43ce-9833-9be42ff16881 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:57.715713 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39840b-8eca-43ce-9833-9be42ff16881 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:57.716087 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39840b-8eca-43ce-9833-9be42ff16881 None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:57.718367 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39840b-8eca-43ce-9833-9be42ff16881 None admin] RBAC: Authorizing `identity:get_project(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:57.719921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39840b-8eca-43ce-9833-9be42ff16881 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:57.721771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 79/149] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:52:57 2022] GET /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:52:57.727642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15da809-5759-4ef2-a769-6945f2f2bf6c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:57.739831 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39840b-8eca-43ce-9833-9be42ff16881 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:57.808854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:57.845014 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15da809-5759-4ef2-a769-6945f2f2bf6c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:57.854588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39840b-8eca-43ce-9833-9be42ff16881 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:57.892624 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:57.893081 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a2a97-b4be-4737-9ce8-bef3e4aace4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:57.894043 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a2a97-b4be-4737-9ce8-bef3e4aace4e None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:57.894326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a2a97-b4be-4737-9ce8-bef3e4aace4e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:57.894603 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a2a97-b4be-4737-9ce8-bef3e4aace4e None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea/users/31ac558b116c43a28ab5c1050a8b4c42/roles/f70856a334f7479fb00bac40f893e9f0` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:57.900270 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57847e9e-970d-4d21-b2e2-9deb090e5b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:57.900837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a2a97-b4be-4737-9ce8-bef3e4aace4e None admin] RBAC: Authorizing `identity:create_grant(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea, user_id=31ac558b116c43a28ab5c1050a8b4c42, role_id=f70856a334f7479fb00bac40f893e9f0)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:57.901312 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57847e9e-970d-4d21-b2e2-9deb090e5b5e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:57.901601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57847e9e-970d-4d21-b2e2-9deb090e5b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:57.901880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57847e9e-970d-4d21-b2e2-9deb090e5b5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:57.902127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a2a97-b4be-4737-9ce8-bef3e4aace4e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:57.915583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57847e9e-970d-4d21-b2e2-9deb090e5b5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:52:57.923973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-57847e9e-970d-4d21-b2e2-9deb090e5b5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:52:57.923973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:52:57.923973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:52:57.923973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:52:57.923973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:52:57.923973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:52:57.923973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:52:57.923973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:52:57.923973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:52:57.923973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:52:57.923973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:52:57.923973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:52:57.923973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:52:57.923973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:52:57.923973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:52:57.923973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:52:57.924906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:52:57.924906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:52:57.924906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:52:57.924906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:52:57.924906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:52:57.924906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:52:57.924906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:52:57.924906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:52:57.924906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:52:57.924906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:52:57.924906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:52:57.924906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:52:57.924906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:52:57.924906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:52:57.924906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:52:57.924906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:52:57.925674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:52:57.925674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:52:57.925674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:52:57.925674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:52:57.925674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:52:57.925674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:52:57.925674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:52:57.925674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:52:57.925674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:52:57.925674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:52:57.925674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:52:57.925674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:52:57.926946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 71/151] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:52:57 2022] PUT /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea/users/31ac558b116c43a28ab5c1050a8b4c42/roles/f70856a334f7479fb00bac40f893e9f0 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:52:57.928697 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 80/151] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:52:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:52:57.932347 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a2a97-b4be-4737-9ce8-bef3e4aace4e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:58.069131 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a2a97-b4be-4737-9ce8-bef3e4aace4e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:58.097183 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:58.110802 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe686b-3e7c-4bf8-a32f-4badb416076f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:58.111814 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe686b-3e7c-4bf8-a32f-4badb416076f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:58.112293 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe686b-3e7c-4bf8-a32f-4badb416076f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:58.112809 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe686b-3e7c-4bf8-a32f-4badb416076f None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:58.113784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe686b-3e7c-4bf8-a32f-4badb416076f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:58.114278 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe686b-3e7c-4bf8-a32f-4badb416076f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:58.115850 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe686b-3e7c-4bf8-a32f-4badb416076f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:58.128464 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 72/152] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:52:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:52:58.134099 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57847e9e-970d-4d21-b2e2-9deb090e5b5e None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:58.273879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57847e9e-970d-4d21-b2e2-9deb090e5b5e None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:58.309861 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ee7db-7e0c-4fd2-bb9e-fdd48735aa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:58.310787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ee7db-7e0c-4fd2-bb9e-fdd48735aa70 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:58.311069 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ee7db-7e0c-4fd2-bb9e-fdd48735aa70 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:58.311340 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ee7db-7e0c-4fd2-bb9e-fdd48735aa70 None admin] PATH_INFO: `/v3/users/c132d4e24cfd4bd197338f840d4d9eb9` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:58.312565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ee7db-7e0c-4fd2-bb9e-fdd48735aa70 None admin] RBAC: Authorizing `identity:get_user(user_id=c132d4e24cfd4bd197338f840d4d9eb9)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:58.314072 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ee7db-7e0c-4fd2-bb9e-fdd48735aa70 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:58.316998 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 81/153] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:52:58 2022] GET /identity/v3/users/c132d4e24cfd4bd197338f840d4d9eb9 => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:58.321596 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe686b-3e7c-4bf8-a32f-4badb416076f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:58.453978 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe686b-3e7c-4bf8-a32f-4badb416076f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:58.489782 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7478b7-e6f9-4363-8180-5b2a7641ae73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:58.491187 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7478b7-e6f9-4363-8180-5b2a7641ae73 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:58.491873 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7478b7-e6f9-4363-8180-5b2a7641ae73 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:58.492608 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7478b7-e6f9-4363-8180-5b2a7641ae73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:58.568840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:58.584581 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7478b7-e6f9-4363-8180-5b2a7641ae73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:58.586365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7478b7-e6f9-4363-8180-5b2a7641ae73 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:58.686126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 73/154] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:52:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:52:58.691902 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ee7db-7e0c-4fd2-bb9e-fdd48735aa70 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:58.781336 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:58.844445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ee7db-7e0c-4fd2-bb9e-fdd48735aa70 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:58.914037 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef30eb-49dd-4c19-8fee-b9c9c1a96e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:58.914967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef30eb-49dd-4c19-8fee-b9c9c1a96e18 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:58.915240 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef30eb-49dd-4c19-8fee-b9c9c1a96e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:58.915551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef30eb-49dd-4c19-8fee-b9c9c1a96e18 None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:58.916873 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef30eb-49dd-4c19-8fee-b9c9c1a96e18 None admin] RBAC: Authorizing `identity:get_project(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:58.918325 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef30eb-49dd-4c19-8fee-b9c9c1a96e18 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:58.924152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 82/155] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:52:58 2022] GET /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:58.926238 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7478b7-e6f9-4363-8180-5b2a7641ae73 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:59.026589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:59.043357 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7478b7-e6f9-4363-8180-5b2a7641ae73 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:59.078745 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d27c5-010b-4fb6-9c2c-2079794a1ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:59.079749 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d27c5-010b-4fb6-9c2c-2079794a1ede None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:59.080215 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d27c5-010b-4fb6-9c2c-2079794a1ede None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:59.080720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d27c5-010b-4fb6-9c2c-2079794a1ede None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:59.081752 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d27c5-010b-4fb6-9c2c-2079794a1ede None admin] RBAC: Adding query filter params (role.id=b1e13c449fa64b3e88b6832bb767214b, scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea, user.id=c132d4e24cfd4bd197338f840d4d9eb9) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:59.082244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d27c5-010b-4fb6-9c2c-2079794a1ede None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:59.083872 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d27c5-010b-4fb6-9c2c-2079794a1ede None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:59.090781 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 74/156] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:52:58 2022] GET /identity/v3/role_assignments?user.id=c132d4e24cfd4bd197338f840d4d9eb9&scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:59.176358 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e65f3d-5a80-4287-b35b-f9ff7b43d172 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:59.177041 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e65f3d-5a80-4287-b35b-f9ff7b43d172 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:59.177527 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e65f3d-5a80-4287-b35b-f9ff7b43d172 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:59.179148 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 83/157] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:52:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:52:59.185384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb9e61-50ce-4ab8-8ca1-bdf7ebaaba71 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:59.186384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb9e61-50ce-4ab8-8ca1-bdf7ebaaba71 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:59.186845 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb9e61-50ce-4ab8-8ca1-bdf7ebaaba71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:59.218676 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-6bbb9e61-50ce-4ab8-8ca1-bdf7ebaaba71 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:52:59.220219 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6bbb9e61-50ce-4ab8-8ca1-bdf7ebaaba71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:52:59.307453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 75/158] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:52:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:52:59.350940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e65f3d-5a80-4287-b35b-f9ff7b43d172 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:59.492017 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e65f3d-5a80-4287-b35b-f9ff7b43d172 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:59.519132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:59.538059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca9f67-a693-48af-a70d-58c61a306ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:59.539243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca9f67-a693-48af-a70d-58c61a306ebf None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:59.539752 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca9f67-a693-48af-a70d-58c61a306ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:59.540217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca9f67-a693-48af-a70d-58c61a306ebf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:59.552831 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca9f67-a693-48af-a70d-58c61a306ebf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:52:59.561275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-7cca9f67-a693-48af-a70d-58c61a306ebf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:52:59.561275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:52:59.561275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:52:59.561275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:52:59.561275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:52:59.561275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:52:59.561275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:52:59.561275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:52:59.561275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:52:59.561275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:52:59.561275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:52:59.561275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:52:59.561275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:52:59.561275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:52:59.561275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:52:59.561275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:52:59.562229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:52:59.562229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:52:59.562229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:52:59.562229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:52:59.562229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:52:59.562229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:52:59.562229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:52:59.562229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:52:59.562229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:52:59.562229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:52:59.562229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:52:59.562229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:52:59.562229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:52:59.562229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:52:59.562229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:52:59.562229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:52:59.563005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:52:59.563005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:52:59.563005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:52:59.563005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:52:59.563005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:52:59.563005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:52:59.563005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:52:59.563005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:52:59.563005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:52:59.563005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:52:59.563005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:52:59.563005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:52:59.569559 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 84/159] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:52:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:52:59.575535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb9e61-50ce-4ab8-8ca1-bdf7ebaaba71 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:59.676438 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:52:59.705938 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb9e61-50ce-4ab8-8ca1-bdf7ebaaba71 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:59.747061 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0530112-4dd9-42e2-9bae-34405911af10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:59.748076 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0530112-4dd9-42e2-9bae-34405911af10 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:59.748569 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0530112-4dd9-42e2-9bae-34405911af10 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:59.749025 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0530112-4dd9-42e2-9bae-34405911af10 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:59.749968 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0530112-4dd9-42e2-9bae-34405911af10 None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:52:59.750494 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0530112-4dd9-42e2-9bae-34405911af10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:59.752075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0530112-4dd9-42e2-9bae-34405911af10 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:59.761832 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 76/160] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:52:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:59.769502 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca9f67-a693-48af-a70d-58c61a306ebf None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:52:59.919445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca9f67-a693-48af-a70d-58c61a306ebf None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:52:59.956973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad3b70-7020-4441-92a1-fb25d8d4995d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:52:59.957856 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad3b70-7020-4441-92a1-fb25d8d4995d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:52:59.958140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad3b70-7020-4441-92a1-fb25d8d4995d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:52:59.958421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad3b70-7020-4441-92a1-fb25d8d4995d None admin] PATH_INFO: `/v3/users/0233db48571f40ee92089c7c9fc98814` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:52:59.965163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad3b70-7020-4441-92a1-fb25d8d4995d None admin] RBAC: Authorizing `identity:get_user(user_id=0233db48571f40ee92089c7c9fc98814)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:52:59.966658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad3b70-7020-4441-92a1-fb25d8d4995d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:52:59.968360 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 85/161] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:52:59 2022] GET /identity/v3/users/0233db48571f40ee92089c7c9fc98814 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:52:59.973992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0530112-4dd9-42e2-9bae-34405911af10 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:00.027467 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fc0ef-365b-420b-841b-8a8acf2d89bb None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:00.027787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fc0ef-365b-420b-841b-8a8acf2d89bb None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:00.028039 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fc0ef-365b-420b-841b-8a8acf2d89bb None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:00.029476 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 86/162] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:00.035028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5dca30-cf95-42c3-bd4f-5dfd7d22a076 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:00.035246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5dca30-cf95-42c3-bd4f-5dfd7d22a076 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:00.035484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5dca30-cf95-42c3-bd4f-5dfd7d22a076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:00.065544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5a5dca30-cf95-42c3-bd4f-5dfd7d22a076 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:00.066946 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5a5dca30-cf95-42c3-bd4f-5dfd7d22a076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:00.112166 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0530112-4dd9-42e2-9bae-34405911af10 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:00.151931 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7dc364-b263-49ae-a5cb-c9d5d7da2d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:00.152650 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7dc364-b263-49ae-a5cb-c9d5d7da2d80 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:00.152922 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7dc364-b263-49ae-a5cb-c9d5d7da2d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:00.153198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7dc364-b263-49ae-a5cb-c9d5d7da2d80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:00.159597 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:00.174758 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:00.226810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 87/163] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:53:00.252103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59252aee-e353-42f3-b114-3d0ecae5549d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:00.252441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59252aee-e353-42f3-b114-3d0ecae5549d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:00.252714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59252aee-e353-42f3-b114-3d0ecae5549d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:00.254277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 88/164] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:53:00.261021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdef08-789a-4aaf-b61f-8e55f6185221 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:00.261309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdef08-789a-4aaf-b61f-8e55f6185221 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:00.261594 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdef08-789a-4aaf-b61f-8e55f6185221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:00.293434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-70cdef08-789a-4aaf-b61f-8e55f6185221 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:00.294640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-70cdef08-789a-4aaf-b61f-8e55f6185221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:00.296192 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7dc364-b263-49ae-a5cb-c9d5d7da2d80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:00.297705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7dc364-b263-49ae-a5cb-c9d5d7da2d80 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:00.446438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 89/165] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:53:00.449257 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cdef08-789a-4aaf-b61f-8e55f6185221 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:00.532642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 77/166] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:52:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:00.532642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7dc364-b263-49ae-a5cb-c9d5d7da2d80 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:00.581153 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:00.635944 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7dc364-b263-49ae-a5cb-c9d5d7da2d80 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:00.654342 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cdef08-789a-4aaf-b61f-8e55f6185221 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:00.670372 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:00.671504 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb354a3-d3ad-49d5-af29-2f94ff8ee106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:00.672327 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb354a3-d3ad-49d5-af29-2f94ff8ee106 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:00.672859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb354a3-d3ad-49d5-af29-2f94ff8ee106 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:00.672960 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb354a3-d3ad-49d5-af29-2f94ff8ee106 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:00.684301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb354a3-d3ad-49d5-af29-2f94ff8ee106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:00.692384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-feb354a3-d3ad-49d5-af29-2f94ff8ee106 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:00.692384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:00.692384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:00.692384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:00.692384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:00.692384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:00.692384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:00.692384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:00.692384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:00.692384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:00.692384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:00.692384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:00.692384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:00.692384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:00.692384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:00.692384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:00.693254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:00.693254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:00.693254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:00.693254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:00.693254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:00.693254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:00.693254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:00.693254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:00.693254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:00.693254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:00.693254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:00.693254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:00.693254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:00.693254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:00.693254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:00.693254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:00.694260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:00.694260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:00.694260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:00.694260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:00.694260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:00.694260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:00.694260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:00.694260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:00.694260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:00.694260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:00.694260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:00.694260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:00.698243 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 78/167] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:53:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:00.698990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb354a3-d3ad-49d5-af29-2f94ff8ee106 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:00.769119 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3ff53-85a8-41f8-a58e-9be3c8f59702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:00.769940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3ff53-85a8-41f8-a58e-9be3c8f59702 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:00.770230 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3ff53-85a8-41f8-a58e-9be3c8f59702 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:00.770510 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3ff53-85a8-41f8-a58e-9be3c8f59702 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:00.807624 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca3ff53-85a8-41f8-a58e-9be3c8f59702 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:00.808294 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:00.816677 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb354a3-d3ad-49d5-af29-2f94ff8ee106 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:00.821655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-cca3ff53-85a8-41f8-a58e-9be3c8f59702 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:00.821655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:00.821655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:00.821655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:00.821655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:00.821655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:00.821655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:00.821655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:00.821655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:00.821655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:00.821655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:00.821655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:00.821655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:00.821655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:00.821655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:00.821655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:00.822551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:00.822551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:00.822551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:00.822551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:00.822551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:00.822551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:00.822551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:00.822551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:00.822551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:00.822551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:00.822551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:00.822551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:00.822551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:00.822551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:00.822551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:00.822551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:00.823314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:00.823314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:00.823314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:00.823314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:00.823314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:00.823314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:00.823314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:00.823314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:00.823314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:00.823314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:00.823314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:00.823314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:00.823314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 90/168] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:00.824036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3ff53-85a8-41f8-a58e-9be3c8f59702 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:00.874225 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ab93f-2d71-4a27-8f65-f8c21552d544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:00.875219 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ab93f-2d71-4a27-8f65-f8c21552d544 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:00.875503 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ab93f-2d71-4a27-8f65-f8c21552d544 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:00.875776 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ab93f-2d71-4a27-8f65-f8c21552d544 None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:00.877058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ab93f-2d71-4a27-8f65-f8c21552d544 None admin] RBAC: Authorizing `identity:get_project(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:00.888994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ab93f-2d71-4a27-8f65-f8c21552d544 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:00.890985 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 79/169] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:00 2022] GET /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:00.892085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ab93f-2d71-4a27-8f65-f8c21552d544 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:01.028710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ab93f-2d71-4a27-8f65-f8c21552d544 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:01.064599 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf04e2-69b1-42c3-b009-947ab80313b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:01.065279 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf04e2-69b1-42c3-b009-947ab80313b0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:01.065557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf04e2-69b1-42c3-b009-947ab80313b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:01.065933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf04e2-69b1-42c3-b009-947ab80313b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:01.066697 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf04e2-69b1-42c3-b009-947ab80313b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:01.067001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf04e2-69b1-42c3-b009-947ab80313b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:01.068394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf04e2-69b1-42c3-b009-947ab80313b0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:01.072244 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3ff53-85a8-41f8-a58e-9be3c8f59702 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:01.077943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 80/170] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:01.078770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf04e2-69b1-42c3-b009-947ab80313b0 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:01.112802 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:01.139826 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21677ab0-2aa5-443a-9ee9-a241680f8094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:01.140831 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21677ab0-2aa5-443a-9ee9-a241680f8094 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:01.141260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21677ab0-2aa5-443a-9ee9-a241680f8094 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:01.142098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21677ab0-2aa5-443a-9ee9-a241680f8094 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:01.143099 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21677ab0-2aa5-443a-9ee9-a241680f8094 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:01.143548 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21677ab0-2aa5-443a-9ee9-a241680f8094 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:01.145216 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21677ab0-2aa5-443a-9ee9-a241680f8094 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:01.162999 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 91/171] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:01.164228 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21677ab0-2aa5-443a-9ee9-a241680f8094 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:01.185933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf04e2-69b1-42c3-b009-947ab80313b0 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:01.222191 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668c467b-8a24-4918-be60-5ad2e623ff8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:01.223128 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c467b-8a24-4918-be60-5ad2e623ff8f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:01.223549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c467b-8a24-4918-be60-5ad2e623ff8f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:01.224008 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:01.225111 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c467b-8a24-4918-be60-5ad2e623ff8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:01.226169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668c467b-8a24-4918-be60-5ad2e623ff8f None admin] RBAC: Adding query filter params (role.id=8b9bae8f81854757b4f389d3bd22f595, scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea, user.id=0233db48571f40ee92089c7c9fc98814) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:01.226704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668c467b-8a24-4918-be60-5ad2e623ff8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:01.229146 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668c467b-8a24-4918-be60-5ad2e623ff8f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:01.236071 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 81/172] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:53:01 2022] GET /identity/v3/role_assignments?user.id=0233db48571f40ee92089c7c9fc98814&scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea&role.id=8b9bae8f81854757b4f389d3bd22f595 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:01.241589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668c467b-8a24-4918-be60-5ad2e623ff8f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:01.378349 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:01.383840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21677ab0-2aa5-443a-9ee9-a241680f8094 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:01.391277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668c467b-8a24-4918-be60-5ad2e623ff8f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:01.458714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c01d8-0c00-498a-92d7-603a318fdffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:01.460340 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c01d8-0c00-498a-92d7-603a318fdffc None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:01.461772 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c01d8-0c00-498a-92d7-603a318fdffc None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:01.462582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c01d8-0c00-498a-92d7-603a318fdffc None admin] PATH_INFO: `/v3/users/0233db48571f40ee92089c7c9fc98814` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:01.465139 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c01d8-0c00-498a-92d7-603a318fdffc None admin] RBAC: Authorizing `identity:get_user(user_id=0233db48571f40ee92089c7c9fc98814)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:01.465634 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482457e3-4a02-4265-ac52-197a71c83643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:01.466764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482457e3-4a02-4265-ac52-197a71c83643 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:01.468543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c01d8-0c00-498a-92d7-603a318fdffc None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:01.469086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482457e3-4a02-4265-ac52-197a71c83643 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:01.469565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482457e3-4a02-4265-ac52-197a71c83643 None admin] PATH_INFO: `/v3/users/0233db48571f40ee92089c7c9fc98814` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:01.472236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 92/173] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:01 2022] GET /identity/v3/users/0233db48571f40ee92089c7c9fc98814 => generated 275 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:01.473143 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482457e3-4a02-4265-ac52-197a71c83643 None admin] RBAC: Authorizing `identity:get_user(user_id=0233db48571f40ee92089c7c9fc98814)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:01.474815 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482457e3-4a02-4265-ac52-197a71c83643 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:01.477197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 82/174] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:01 2022] GET /identity/v3/users/0233db48571f40ee92089c7c9fc98814 => generated 275 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:01.480209 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c01d8-0c00-498a-92d7-603a318fdffc None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:01.482058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482457e3-4a02-4265-ac52-197a71c83643 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:01.651233 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:01.655065 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c01d8-0c00-498a-92d7-603a318fdffc None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:01.662002 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482457e3-4a02-4265-ac52-197a71c83643 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:01.690340 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ce18e-c411-4de4-a49b-c83a67dfce8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:01.691268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ce18e-c411-4de4-a49b-c83a67dfce8d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:01.691698 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ce18e-c411-4de4-a49b-c83a67dfce8d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:01.692111 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ce18e-c411-4de4-a49b-c83a67dfce8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:01.701314 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b98ff-4443-467b-83f8-63c28e60ecdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:01.702420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b98ff-4443-467b-83f8-63c28e60ecdd None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:01.702919 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b98ff-4443-467b-83f8-63c28e60ecdd None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:01.703690 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b98ff-4443-467b-83f8-63c28e60ecdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:01.738604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:01.767081 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ce18e-c411-4de4-a49b-c83a67dfce8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:01.768574 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ce18e-c411-4de4-a49b-c83a67dfce8d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:01.783966 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b98ff-4443-467b-83f8-63c28e60ecdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:01.785899 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b98ff-4443-467b-83f8-63c28e60ecdd None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:01.871132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:01.875827 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 93/175] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 25 15:53:01.877716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a3bb5-a76a-4722-bd41-a323f2a9a826 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:01.878550 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a3bb5-a76a-4722-bd41-a323f2a9a826 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:01.879235 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a3bb5-a76a-4722-bd41-a323f2a9a826 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:01.880801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 94/176] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:01.881887 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774699e-c174-49da-8e59-fa24412db8ea None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:01.882176 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774699e-c174-49da-8e59-fa24412db8ea None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:01.882456 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774699e-c174-49da-8e59-fa24412db8ea None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:01.883790 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 95/177] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:01.884822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a1d2e-f50d-469b-95e4-f1391b496d23 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:01.885106 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a1d2e-f50d-469b-95e4-f1391b496d23 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:01.885565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a1d2e-f50d-469b-95e4-f1391b496d23 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:01.886938 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 96/178] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:01.887501 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a1d2e-f50d-469b-95e4-f1391b496d23 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:01.903199 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 83/179] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:01.904834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40c53a-1e67-4eb8-8929-8e811bf89a4e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:01.905391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40c53a-1e67-4eb8-8929-8e811bf89a4e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:01.905878 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af40c53a-1e67-4eb8-8929-8e811bf89a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:01.923051 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-af40c53a-1e67-4eb8-8929-8e811bf89a4e None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:01.924494 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-af40c53a-1e67-4eb8-8929-8e811bf89a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:01.982447 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:01.991839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a1d2e-f50d-469b-95e4-f1391b496d23 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:01.993080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 84/180] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:01.994491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c73d4-71b5-4c81-acb5-72cce3c10130 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:01.994782 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c73d4-71b5-4c81-acb5-72cce3c10130 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:01.995114 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c73d4-71b5-4c81-acb5-72cce3c10130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:02.011125 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c63c73d4-71b5-4c81-acb5-72cce3c10130 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:02.012314 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c63c73d4-71b5-4c81-acb5-72cce3c10130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:02.027877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd24d8-5ebc-407a-897d-be4f962b1af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:02.028801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd24d8-5ebc-407a-897d-be4f962b1af2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:02.029085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd24d8-5ebc-407a-897d-be4f962b1af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:02.029364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd24d8-5ebc-407a-897d-be4f962b1af2 None admin] PATH_INFO: `/v3/projects/7fda594bccc94e469c094a50537682fa` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:02.030560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd24d8-5ebc-407a-897d-be4f962b1af2 None admin] RBAC: Authorizing `identity:get_project(project_id=7fda594bccc94e469c094a50537682fa)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:02.032008 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd24d8-5ebc-407a-897d-be4f962b1af2 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:02.033758 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 97/181] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:01 2022] GET /identity/v3/projects/7fda594bccc94e469c094a50537682fa => generated 312 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:02.034877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf2640-53a2-4088-92d6-73a3a252145a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:02.035309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf2640-53a2-4088-92d6-73a3a252145a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:02.035748 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf2640-53a2-4088-92d6-73a3a252145a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:02.051643 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-19bf2640-53a2-4088-92d6-73a3a252145a None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:02.052835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-19bf2640-53a2-4088-92d6-73a3a252145a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:02.080428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 85/182] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:02.081344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c73d4-71b5-4c81-acb5-72cce3c10130 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:02.115221 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:02.123347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 98/183] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:02.124708 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec76814-fd24-4b6b-b565-20edfdc0c42e None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:02.124995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec76814-fd24-4b6b-b565-20edfdc0c42e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:02.125275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec76814-fd24-4b6b-b565-20edfdc0c42e None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:02.126552 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 99/184] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:02.127243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec76814-fd24-4b6b-b565-20edfdc0c42e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:02.186106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c73d4-71b5-4c81-acb5-72cce3c10130 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:02.208598 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:02.222199 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df9ac9-b080-45c7-a79b-8fbd9963c22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:02.223067 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df9ac9-b080-45c7-a79b-8fbd9963c22b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:02.223340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df9ac9-b080-45c7-a79b-8fbd9963c22b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:02.223619 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df9ac9-b080-45c7-a79b-8fbd9963c22b None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:02.224862 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df9ac9-b080-45c7-a79b-8fbd9963c22b None admin] RBAC: Authorizing `identity:get_project(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:02.226365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df9ac9-b080-45c7-a79b-8fbd9963c22b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:02.228059 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 86/185] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:02 2022] GET /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:02.228299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec76814-fd24-4b6b-b565-20edfdc0c42e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:02.233327 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df9ac9-b080-45c7-a79b-8fbd9963c22b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:02.264154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2882833-6d23-41ef-b26c-51a1d3f8c753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:02.264966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2882833-6d23-41ef-b26c-51a1d3f8c753 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:02.265243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2882833-6d23-41ef-b26c-51a1d3f8c753 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:02.265513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2882833-6d23-41ef-b26c-51a1d3f8c753 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:02.279273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2882833-6d23-41ef-b26c-51a1d3f8c753 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:02.288249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-d2882833-6d23-41ef-b26c-51a1d3f8c753 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 25 15:53:02.288249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:02.288249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:02.288249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:02.288249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:02.288249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:02.288249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:02.288249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:02.288249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:02.288249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:02.288249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:02.288249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:02.288249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:02.288249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:02.288249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:02.288249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:02.289189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:02.289189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:02.289189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:02.289189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:02.289189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:02.289189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:02.289189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:02.289189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:02.289189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:02.289189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:02.289189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:02.289189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:02.289189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:02.289189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:02.289189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:02.289189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:02.289966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:02.289966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:02.289966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:02.289966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:02.289966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:02.289966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:02.289966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:02.289966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:02.289966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:02.289966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:02.289966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 25 15:53:02.289966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:02.289966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 100/186] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:02.292233 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2882833-6d23-41ef-b26c-51a1d3f8c753 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:02.330827 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df9ac9-b080-45c7-a79b-8fbd9963c22b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:02.349143 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:02.365421 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f4429-5c41-47fa-b8fe-1e25ca3e3b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:02.366078 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f4429-5c41-47fa-b8fe-1e25ca3e3b30 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:02.366362 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f4429-5c41-47fa-b8fe-1e25ca3e3b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:02.366642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f4429-5c41-47fa-b8fe-1e25ca3e3b30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:02.367476 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f4429-5c41-47fa-b8fe-1e25ca3e3b30 None admin] RBAC: Adding query filter params (role.id=8b9bae8f81854757b4f389d3bd22f595, scope.project.id=7fda594bccc94e469c094a50537682fa, user.id=0233db48571f40ee92089c7c9fc98814) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:02.367786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f4429-5c41-47fa-b8fe-1e25ca3e3b30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:02.369325 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f4429-5c41-47fa-b8fe-1e25ca3e3b30 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:02.375178 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 87/187] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:53:02 2022] GET /identity/v3/role_assignments?user.id=0233db48571f40ee92089c7c9fc98814&scope.project.id=7fda594bccc94e469c094a50537682fa&role.id=8b9bae8f81854757b4f389d3bd22f595 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:02.376326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c6a50-0e34-4874-b35b-36f967903236 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:02.376640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c6a50-0e34-4874-b35b-36f967903236 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:02.376918 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c6a50-0e34-4874-b35b-36f967903236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:02.394508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-276c6a50-0e34-4874-b35b-36f967903236 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:02.395736 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-276c6a50-0e34-4874-b35b-36f967903236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:02.410007 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2882833-6d23-41ef-b26c-51a1d3f8c753 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:02.451527 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f72329-8861-466e-8367-1c431ada1d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:02.452260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f72329-8861-466e-8367-1c431ada1d5d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:02.452635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f72329-8861-466e-8367-1c431ada1d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:02.452764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:02.452942 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f72329-8861-466e-8367-1c431ada1d5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:02.466191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f72329-8861-466e-8367-1c431ada1d5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:02.473890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 88/188] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:02.474567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-76f72329-8861-466e-8367-1c431ada1d5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:53:02.474567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:02.474567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:02.474567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:02.474567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:02.474567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:02.474567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:02.474567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:02.474567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:02.474567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:02.474567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:02.474567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:02.474567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:02.474567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:02.474567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:02.474567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:02.475448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:02.475448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:02.475448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:02.475448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:02.475448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:02.475448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:02.475448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:02.475448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:02.475448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:02.475448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:02.475448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:02.475448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:02.475448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:02.475448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:02.475448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:02.475448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:02.481465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:02.481465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:02.481465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:02.481465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:02.481465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:02.481465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:02.481465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:02.481465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:02.481465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:02.481465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:02.481465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:53:02.481465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:02.482072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c6a50-0e34-4874-b35b-36f967903236 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:02.482142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 101/189] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:53:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:02.482142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f72329-8861-466e-8367-1c431ada1d5d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:02.595843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c6a50-0e34-4874-b35b-36f967903236 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:02.598242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f72329-8861-466e-8367-1c431ada1d5d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:02.616096 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:02.634192 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce71923-4cc0-4533-ac75-1e6b1cb5b5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:02.634802 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cfb672-4297-433a-8fcf-9d6d2ef029f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:02.635086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce71923-4cc0-4533-ac75-1e6b1cb5b5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:02.635365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce71923-4cc0-4533-ac75-1e6b1cb5b5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:02.635486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfb672-4297-433a-8fcf-9d6d2ef029f2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:02.635633 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce71923-4cc0-4533-ac75-1e6b1cb5b5b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:02.635718 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfb672-4297-433a-8fcf-9d6d2ef029f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:02.635985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfb672-4297-433a-8fcf-9d6d2ef029f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:02.636853 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cfb672-4297-433a-8fcf-9d6d2ef029f2 None admin] RBAC: Adding query filter params (role.id=6322fb4a438d4a389c43d05248665e72, scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea, user.id=0233db48571f40ee92089c7c9fc98814) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:02.637163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cfb672-4297-433a-8fcf-9d6d2ef029f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:02.638631 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cfb672-4297-433a-8fcf-9d6d2ef029f2 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:02.644920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 102/190] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:53:02 2022] GET /identity/v3/role_assignments?user.id=0233db48571f40ee92089c7c9fc98814&scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea&role.id=6322fb4a438d4a389c43d05248665e72 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:02.645700 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cfb672-4297-433a-8fcf-9d6d2ef029f2 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:02.646619 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce71923-4cc0-4533-ac75-1e6b1cb5b5b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:02.655248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-6ce71923-4cc0-4533-ac75-1e6b1cb5b5b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:02.655248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:02.655248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:02.655248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:02.655248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:02.655248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:02.655248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:02.655248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:02.655248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:02.655248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:02.655248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:02.655248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:02.655248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:02.655248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:02.655248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:02.655248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:02.656198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:02.656198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:02.656198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:02.656198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:02.656198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:02.656198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:02.656198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:02.656198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:02.656198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:02.656198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:02.656198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:02.656198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:02.656198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:02.656198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:02.656198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:02.656198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:02.657372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:02.657372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:02.657372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:02.657372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:02.657372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:02.657372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:02.657372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:02.657372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:02.657372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:02.657372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:02.657372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:02.657372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:02.657372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 89/191] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:53:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:02.658015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce71923-4cc0-4533-ac75-1e6b1cb5b5b5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:02.716993 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:02.770855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cfb672-4297-433a-8fcf-9d6d2ef029f2 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:02.781227 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce71923-4cc0-4533-ac75-1e6b1cb5b5b5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:02.812220 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b67c6b-8bd7-46ca-9327-c5911aef56d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:02.812922 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b67c6b-8bd7-46ca-9327-c5911aef56d5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:02.813204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b67c6b-8bd7-46ca-9327-c5911aef56d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:02.813484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b67c6b-8bd7-46ca-9327-c5911aef56d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:02.814245 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b67c6b-8bd7-46ca-9327-c5911aef56d5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:02.814533 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b67c6b-8bd7-46ca-9327-c5911aef56d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:02.815920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b67c6b-8bd7-46ca-9327-c5911aef56d5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:02.819518 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d6e41-1d09-4270-8da1-a4ed7cc5533b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:02.820170 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d6e41-1d09-4270-8da1-a4ed7cc5533b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:02.820491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d6e41-1d09-4270-8da1-a4ed7cc5533b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:02.820736 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d6e41-1d09-4270-8da1-a4ed7cc5533b None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:02.821493 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d6e41-1d09-4270-8da1-a4ed7cc5533b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:02.821857 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d6e41-1d09-4270-8da1-a4ed7cc5533b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:02.823370 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d6e41-1d09-4270-8da1-a4ed7cc5533b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:02.825441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 103/192] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:02.826348 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b67c6b-8bd7-46ca-9327-c5911aef56d5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:02.838307 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 90/193] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:02.839391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d6e41-1d09-4270-8da1-a4ed7cc5533b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:02.874706 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:02.939539 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b67c6b-8bd7-46ca-9327-c5911aef56d5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:02.947959 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d6e41-1d09-4270-8da1-a4ed7cc5533b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:02.955410 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:02.979889 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7daef-f2e9-4bb2-8656-f747165540e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:02.980651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7daef-f2e9-4bb2-8656-f747165540e4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:02.980931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7daef-f2e9-4bb2-8656-f747165540e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:02.981207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7daef-f2e9-4bb2-8656-f747165540e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:02.982416 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed7daef-f2e9-4bb2-8656-f747165540e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:02.982933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc238b44-4210-4747-a131-10942f7709cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:02.983592 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc238b44-4210-4747-a131-10942f7709cf None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:02.983901 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc238b44-4210-4747-a131-10942f7709cf None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:02.984029 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed7daef-f2e9-4bb2-8656-f747165540e4 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:02.984140 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc238b44-4210-4747-a131-10942f7709cf None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:02.984941 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc238b44-4210-4747-a131-10942f7709cf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:02.985241 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc238b44-4210-4747-a131-10942f7709cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:02.985557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 104/194] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:02.986305 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7daef-f2e9-4bb2-8656-f747165540e4 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:02.986651 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc238b44-4210-4747-a131-10942f7709cf None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:02.995742 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 91/195] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:02.996620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc238b44-4210-4747-a131-10942f7709cf None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:03.083419 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7daef-f2e9-4bb2-8656-f747165540e4 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:03.093433 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:03.093678 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc238b44-4210-4747-a131-10942f7709cf None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:03.117728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49e7d0-ea0c-4274-ab60-d7b958a143e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:03.118606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49e7d0-ea0c-4274-ab60-d7b958a143e0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:03.118918 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49e7d0-ea0c-4274-ab60-d7b958a143e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:03.119351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49e7d0-ea0c-4274-ab60-d7b958a143e0 None admin] PATH_INFO: `/v3/users/31ac558b116c43a28ab5c1050a8b4c42` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:03.120819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49e7d0-ea0c-4274-ab60-d7b958a143e0 None admin] RBAC: Authorizing `identity:get_user(user_id=31ac558b116c43a28ab5c1050a8b4c42)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:03.122250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49e7d0-ea0c-4274-ab60-d7b958a143e0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:03.128464 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 105/196] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:02 2022] GET /identity/v3/users/31ac558b116c43a28ab5c1050a8b4c42 => generated 289 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:03.128464 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49e7d0-ea0c-4274-ab60-d7b958a143e0 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:03.128695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd18569-3b2a-4a24-8a64-c4996e3b86c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:03.128695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd18569-3b2a-4a24-8a64-c4996e3b86c6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:03.128695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd18569-3b2a-4a24-8a64-c4996e3b86c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:03.128695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd18569-3b2a-4a24-8a64-c4996e3b86c6 None admin] PATH_INFO: `/v3/users/6525e3c87b6c4fe783339d0d3765d462` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:03.142394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd18569-3b2a-4a24-8a64-c4996e3b86c6 None admin] RBAC: Authorizing `identity:get_user(user_id=6525e3c87b6c4fe783339d0d3765d462)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:03.143870 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd18569-3b2a-4a24-8a64-c4996e3b86c6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:03.145585 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 92/197] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:02 2022] GET /identity/v3/users/6525e3c87b6c4fe783339d0d3765d462 => generated 283 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:03.146333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd18569-3b2a-4a24-8a64-c4996e3b86c6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:03.173935 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:03.230467 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49e7d0-ea0c-4274-ab60-d7b958a143e0 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:03.245082 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd18569-3b2a-4a24-8a64-c4996e3b86c6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:03.262322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc27b1-dec0-4906-9f27-3728d4422d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:03.262993 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc27b1-dec0-4906-9f27-3728d4422d46 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:03.263279 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc27b1-dec0-4906-9f27-3728d4422d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:03.263559 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc27b1-dec0-4906-9f27-3728d4422d46 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:03.264274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfc27b1-dec0-4906-9f27-3728d4422d46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:03.265764 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfc27b1-dec0-4906-9f27-3728d4422d46 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:03.279071 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e56a52-5ac5-48a5-82fd-792fac107e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:03.279944 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e56a52-5ac5-48a5-82fd-792fac107e52 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:03.280221 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e56a52-5ac5-48a5-82fd-792fac107e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:03.280522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e56a52-5ac5-48a5-82fd-792fac107e52 None admin] PATH_INFO: `/v3/users/6525e3c87b6c4fe783339d0d3765d462` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:03.281745 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e56a52-5ac5-48a5-82fd-792fac107e52 None admin] RBAC: Authorizing `identity:get_user(user_id=6525e3c87b6c4fe783339d0d3765d462)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:03.283197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e56a52-5ac5-48a5-82fd-792fac107e52 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:03.289872 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 93/198] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:03 2022] GET /identity/v3/users/6525e3c87b6c4fe783339d0d3765d462 => generated 283 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:03.289872 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e56a52-5ac5-48a5-82fd-792fac107e52 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:03.290107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 106/199] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Jun 25 15:53:03 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:53:03.290107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc27b1-dec0-4906-9f27-3728d4422d46 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:03.327590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:03.397827 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:03.404562 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc27b1-dec0-4906-9f27-3728d4422d46 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:03.406187 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e56a52-5ac5-48a5-82fd-792fac107e52 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:03.441806 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fcf99-08e2-45e9-a310-14112e95e31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:03.442469 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fcf99-08e2-45e9-a310-14112e95e31b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:03.442748 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fcf99-08e2-45e9-a310-14112e95e31b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:03.443024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fcf99-08e2-45e9-a310-14112e95e31b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:03.443635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870aad6-4c95-4042-a284-8da8cfa300c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:03.444647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870aad6-4c95-4042-a284-8da8cfa300c4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:03.444928 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870aad6-4c95-4042-a284-8da8cfa300c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:03.445203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870aad6-4c95-4042-a284-8da8cfa300c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:03.524385 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084fcf99-08e2-45e9-a310-14112e95e31b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:03.525493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870aad6-4c95-4042-a284-8da8cfa300c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:03.526060 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084fcf99-08e2-45e9-a310-14112e95e31b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:03.527071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870aad6-4c95-4042-a284-8da8cfa300c4 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:03.575893 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:03.636525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:03.645042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 94/200] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:03.645960 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fcf99-08e2-45e9-a310-14112e95e31b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:03.665339 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 107/201] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:03.666362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870aad6-4c95-4042-a284-8da8cfa300c4 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:03.760493 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fcf99-08e2-45e9-a310-14112e95e31b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:03.786348 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870aad6-4c95-4042-a284-8da8cfa300c4 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:03.801997 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901811a-327e-4e66-86bc-a29b46ffa60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:03.802988 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901811a-327e-4e66-86bc-a29b46ffa60f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:03.803728 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901811a-327e-4e66-86bc-a29b46ffa60f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:03.804572 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901811a-327e-4e66-86bc-a29b46ffa60f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:03.826483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5389817-8af8-46b6-ba06-c4c9f710640f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:03.827681 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5389817-8af8-46b6-ba06-c4c9f710640f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:03.828215 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5389817-8af8-46b6-ba06-c4c9f710640f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:03.828521 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5389817-8af8-46b6-ba06-c4c9f710640f None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:03.829736 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5389817-8af8-46b6-ba06-c4c9f710640f None admin] RBAC: Authorizing `identity:get_project(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:03.831631 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5389817-8af8-46b6-ba06-c4c9f710640f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:03.833836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 108/202] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:03 2022] GET /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:03.834693 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5389817-8af8-46b6-ba06-c4c9f710640f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:03.838253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:03.896016 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901811a-327e-4e66-86bc-a29b46ffa60f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:03.897570 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901811a-327e-4e66-86bc-a29b46ffa60f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:03.900766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:03.961919 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5389817-8af8-46b6-ba06-c4c9f710640f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:03.999810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f7436-a500-43fc-a5f0-29faf8a27326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:04.000711 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f7436-a500-43fc-a5f0-29faf8a27326 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:04.000986 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f7436-a500-43fc-a5f0-29faf8a27326 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:04.001259 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f7436-a500-43fc-a5f0-29faf8a27326 None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:04.002452 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f7436-a500-43fc-a5f0-29faf8a27326 None admin] RBAC: Authorizing `identity:get_project(project_id=3233ec59a0994511b0032830b4ded6c2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:04.003901 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f7436-a500-43fc-a5f0-29faf8a27326 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:04.006011 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 109/203] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:03 2022] GET /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:04.006666 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f7436-a500-43fc-a5f0-29faf8a27326 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:04.024210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 95/204] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:04.025211 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901811a-327e-4e66-86bc-a29b46ffa60f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:04.089818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:04.125583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f7436-a500-43fc-a5f0-29faf8a27326 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:04.151364 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:04.162276 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901811a-327e-4e66-86bc-a29b46ffa60f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:04.165489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a9de6-c742-4d36-b7d9-bd7ca74db004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:04.165489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a9de6-c742-4d36-b7d9-bd7ca74db004 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:04.165741 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a9de6-c742-4d36-b7d9-bd7ca74db004 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:04.166098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a9de6-c742-4d36-b7d9-bd7ca74db004 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:04.167663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a9de6-c742-4d36-b7d9-bd7ca74db004 None admin] RBAC: Adding query filter params (role.id=f70856a334f7479fb00bac40f893e9f0, scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea, user.id=31ac558b116c43a28ab5c1050a8b4c42) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:04.167981 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a9de6-c742-4d36-b7d9-bd7ca74db004 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:04.169505 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a9de6-c742-4d36-b7d9-bd7ca74db004 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:04.176268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 110/205] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:53:04 2022] GET /identity/v3/role_assignments?user.id=31ac558b116c43a28ab5c1050a8b4c42&scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea&role.id=f70856a334f7479fb00bac40f893e9f0 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:04.177486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a9de6-c742-4d36-b7d9-bd7ca74db004 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:04.208235 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b12c50-790c-4311-9e60-6306a841784f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:04.209461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b12c50-790c-4311-9e60-6306a841784f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:04.209890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b12c50-790c-4311-9e60-6306a841784f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:04.210573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b12c50-790c-4311-9e60-6306a841784f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:04.212131 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b12c50-790c-4311-9e60-6306a841784f None admin] RBAC: Adding query filter params (role.id=b1e13c449fa64b3e88b6832bb767214b, scope.project.id=3233ec59a0994511b0032830b4ded6c2, user.id=6525e3c87b6c4fe783339d0d3765d462) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:04.213241 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b12c50-790c-4311-9e60-6306a841784f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:04.215329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b12c50-790c-4311-9e60-6306a841784f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:04.226954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 96/206] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:53:04 2022] GET /identity/v3/role_assignments?user.id=6525e3c87b6c4fe783339d0d3765d462&scope.project.id=3233ec59a0994511b0032830b4ded6c2&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:04.306206 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a9de6-c742-4d36-b7d9-bd7ca74db004 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:04.349652 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c87c8-b60e-4652-b8f5-19e997ad3be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:04.350469 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c87c8-b60e-4652-b8f5-19e997ad3be2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:04.350754 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c87c8-b60e-4652-b8f5-19e997ad3be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:04.351033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c87c8-b60e-4652-b8f5-19e997ad3be2 None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:04.352337 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c87c8-b60e-4652-b8f5-19e997ad3be2 None admin] RBAC: Authorizing `identity:get_project(project_id=3233ec59a0994511b0032830b4ded6c2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:04.354205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c87c8-b60e-4652-b8f5-19e997ad3be2 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:04.360060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 111/207] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:04 2022] GET /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:04.362834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b12c50-790c-4311-9e60-6306a841784f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:04.477913 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b12c50-790c-4311-9e60-6306a841784f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:04.515463 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7138d6-de7b-4486-bcb4-ad5918752e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:04.516129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7138d6-de7b-4486-bcb4-ad5918752e8e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:04.516740 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7138d6-de7b-4486-bcb4-ad5918752e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:04.517062 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7138d6-de7b-4486-bcb4-ad5918752e8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:04.517908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7138d6-de7b-4486-bcb4-ad5918752e8e None admin] RBAC: Adding query filter params (role.id=6322fb4a438d4a389c43d05248665e72, scope.project.id=3233ec59a0994511b0032830b4ded6c2, user.id=6525e3c87b6c4fe783339d0d3765d462) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:04.518218 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7138d6-de7b-4486-bcb4-ad5918752e8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:04.519678 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7138d6-de7b-4486-bcb4-ad5918752e8e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:04.526258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 97/208] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:53:04 2022] GET /identity/v3/role_assignments?user.id=6525e3c87b6c4fe783339d0d3765d462&scope.project.id=3233ec59a0994511b0032830b4ded6c2&role.id=6322fb4a438d4a389c43d05248665e72 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:05.559602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97526c-d230-4fd2-bcb8-9d31fb4e96e6 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:05.560147 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97526c-d230-4fd2-bcb8-9d31fb4e96e6 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:05.560147 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97526c-d230-4fd2-bcb8-9d31fb4e96e6 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:05.561593 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 112/209] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:05.566920 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c61aab-881b-4147-b0da-4c2da6177b17 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:05.567314 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c61aab-881b-4147-b0da-4c2da6177b17 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:05.567475 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c61aab-881b-4147-b0da-4c2da6177b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:05.585513 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-85c61aab-881b-4147-b0da-4c2da6177b17 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:05.586717 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-85c61aab-881b-4147-b0da-4c2da6177b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:05.604129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:05.621923 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ebecbf-84af-43f2-9374-12b5e003ac96 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:05.622202 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ebecbf-84af-43f2-9374-12b5e003ac96 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:05.622477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ebecbf-84af-43f2-9374-12b5e003ac96 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:05.624120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 113/210] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:05.629050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0bbf3-56f3-4193-9817-0f03c20414f6 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:05.629342 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0bbf3-56f3-4193-9817-0f03c20414f6 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:05.629620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0bbf3-56f3-4193-9817-0f03c20414f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:05.635147 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:05.648773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3eb0bbf3-56f3-4193-9817-0f03c20414f6 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:05.650079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3eb0bbf3-56f3-4193-9817-0f03c20414f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:05.657352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 98/211] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:53:05.680674 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c61aab-881b-4147-b0da-4c2da6177b17 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:05.733511 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 114/212] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:05.757480 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0bbf3-56f3-4193-9817-0f03c20414f6 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:05.788671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c61aab-881b-4147-b0da-4c2da6177b17 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:05.826053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012fa49-2276-40f0-a346-bfdb472650cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:05.826776 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012fa49-2276-40f0-a346-bfdb472650cb None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:05.827157 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012fa49-2276-40f0-a346-bfdb472650cb None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:05.827441 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3012fa49-2276-40f0-a346-bfdb472650cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:05.838621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3012fa49-2276-40f0-a346-bfdb472650cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:05.846675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-3012fa49-2276-40f0-a346-bfdb472650cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:05.846675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:05.846675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:05.846675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:05.846675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:05.846675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:05.846675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:05.846675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:05.846675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:05.846675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:05.846675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:05.846675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:05.846675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:05.846675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:05.846675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:05.846675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:05.847563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:05.847563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:05.847563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:05.847563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:05.847563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:05.847563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:05.847563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:05.847563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:05.847563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:05.847563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:05.847563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:05.847563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:05.847563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:05.847563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:05.847563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:05.847563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:05.848334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:05.848334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:05.848334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:05.848334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:05.848334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:05.848334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:05.848334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:05.848334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:05.848334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:05.848334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:05.848334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:05.848334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:05.848334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 99/213] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:05.859406 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012fa49-2276-40f0-a346-bfdb472650cb None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:05.866494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0bbf3-56f3-4193-9817-0f03c20414f6 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:05.880339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:05.897167 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:05.902966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90457b20-474c-4b3e-8157-cfdedf1bf668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:05.904024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90457b20-474c-4b3e-8157-cfdedf1bf668 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:05.904618 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90457b20-474c-4b3e-8157-cfdedf1bf668 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:05.904994 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90457b20-474c-4b3e-8157-cfdedf1bf668 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:05.906322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90457b20-474c-4b3e-8157-cfdedf1bf668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:05.908037 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90457b20-474c-4b3e-8157-cfdedf1bf668 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:05.914081 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 115/214] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:05.916527 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90457b20-474c-4b3e-8157-cfdedf1bf668 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:05.979977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3012fa49-2276-40f0-a346-bfdb472650cb None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:06.014856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c36c8-d08d-4cec-beb7-ab9c90e7ab2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:06.015587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c36c8-d08d-4cec-beb7-ab9c90e7ab2d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:06.015869 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c36c8-d08d-4cec-beb7-ab9c90e7ab2d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:06.016147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c36c8-d08d-4cec-beb7-ab9c90e7ab2d None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:06.017285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c36c8-d08d-4cec-beb7-ab9c90e7ab2d None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:06.017594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c36c8-d08d-4cec-beb7-ab9c90e7ab2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:06.018578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90457b20-474c-4b3e-8157-cfdedf1bf668 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:06.019000 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c36c8-d08d-4cec-beb7-ab9c90e7ab2d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:06.032342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 100/215] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:06.033191 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c36c8-d08d-4cec-beb7-ab9c90e7ab2d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:06.053743 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fbe3e-7d59-48f9-b0b3-4deaf22e335c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:06.054729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fbe3e-7d59-48f9-b0b3-4deaf22e335c None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:06.055230 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fbe3e-7d59-48f9-b0b3-4deaf22e335c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:06.055698 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fbe3e-7d59-48f9-b0b3-4deaf22e335c None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:06.056742 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053fbe3e-7d59-48f9-b0b3-4deaf22e335c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:06.058614 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053fbe3e-7d59-48f9-b0b3-4deaf22e335c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:06.082149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 116/216] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Jun 25 15:53:05 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:53:06.119926 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:06.149536 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c36c8-d08d-4cec-beb7-ab9c90e7ab2d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:06.188117 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d13c9-371d-4a5f-87f2-a7ba5a9b8edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:06.189075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d13c9-371d-4a5f-87f2-a7ba5a9b8edb None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:06.189355 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d13c9-371d-4a5f-87f2-a7ba5a9b8edb None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:06.189629 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d13c9-371d-4a5f-87f2-a7ba5a9b8edb None admin] PATH_INFO: `/v3/users/5e42b1ec7f9f4871ab17c6f94ddd7f82` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:06.205374 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d13c9-371d-4a5f-87f2-a7ba5a9b8edb None admin] RBAC: Authorizing `identity:get_user(user_id=5e42b1ec7f9f4871ab17c6f94ddd7f82)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:06.206873 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d13c9-371d-4a5f-87f2-a7ba5a9b8edb None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:06.213410 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 101/217] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:06 2022] GET /identity/v3/users/5e42b1ec7f9f4871ab17c6f94ddd7f82 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:06.217586 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fbe3e-7d59-48f9-b0b3-4deaf22e335c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:06.284654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:06.310825 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a85bc9-bbce-4833-a80d-27372f4b0d33 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:06.311141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a85bc9-bbce-4833-a80d-27372f4b0d33 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:06.311382 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a85bc9-bbce-4833-a80d-27372f4b0d33 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:06.313390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 102/218] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:53:06.318611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653b058-fe27-4e9e-bc05-4515bb5aba17 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:06.318905 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653b058-fe27-4e9e-bc05-4515bb5aba17 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:06.319168 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653b058-fe27-4e9e-bc05-4515bb5aba17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:06.334171 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fbe3e-7d59-48f9-b0b3-4deaf22e335c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:06.339851 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c653b058-fe27-4e9e-bc05-4515bb5aba17 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:06.341095 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c653b058-fe27-4e9e-bc05-4515bb5aba17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:06.373056 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749f9c6-ea6b-4295-8534-dd3a047e3667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:06.373724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749f9c6-ea6b-4295-8534-dd3a047e3667 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:06.374006 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749f9c6-ea6b-4295-8534-dd3a047e3667 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:06.374281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749f9c6-ea6b-4295-8534-dd3a047e3667 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:06.434369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 103/219] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:53:06.457212 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653b058-fe27-4e9e-bc05-4515bb5aba17 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:06.468907 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749f9c6-ea6b-4295-8534-dd3a047e3667 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:06.470791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749f9c6-ea6b-4295-8534-dd3a047e3667 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:06.539184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:06.542057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:06.598811 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653b058-fe27-4e9e-bc05-4515bb5aba17 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:06.599984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 117/220] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:06.606498 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749f9c6-ea6b-4295-8534-dd3a047e3667 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:06.648716 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7566c8-af00-4c44-a218-b051abb496a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:06.649717 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7566c8-af00-4c44-a218-b051abb496a6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:06.649913 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7566c8-af00-4c44-a218-b051abb496a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:06.650207 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7566c8-af00-4c44-a218-b051abb496a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:06.661468 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7566c8-af00-4c44-a218-b051abb496a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:06.670058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-3a7566c8-af00-4c44-a218-b051abb496a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:53:06.670058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:06.670058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:06.670058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:06.670058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:06.670058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:06.670058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:06.670058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:06.670058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:06.670058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:06.670058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:06.670058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:06.670058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:06.670058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:06.670058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:06.670058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:06.670952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:06.670952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:06.670952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:06.670952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:06.670952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:06.670952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:06.670952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:06.670952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:06.670952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:06.670952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:06.670952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:06.670952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:06.670952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:06.670952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:06.670952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:06.670952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:06.671743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:06.671743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:06.671743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:06.671743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:06.671743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:06.671743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:06.671743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:06.671743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:06.671743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:06.671743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:06.671743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:53:06.671743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:06.671743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 104/221] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:53:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:06.681591 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7566c8-af00-4c44-a218-b051abb496a6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:06.723830 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749f9c6-ea6b-4295-8534-dd3a047e3667 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:06.761615 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a71a10-d8bf-499b-82ba-e8f5e1becbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:06.763038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a71a10-d8bf-499b-82ba-e8f5e1becbd6 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:06.763552 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a71a10-d8bf-499b-82ba-e8f5e1becbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:06.764034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a71a10-d8bf-499b-82ba-e8f5e1becbd6 None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:06.765581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a71a10-d8bf-499b-82ba-e8f5e1becbd6 None admin] RBAC: Authorizing `identity:get_project(project_id=3233ec59a0994511b0032830b4ded6c2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:06.767628 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a71a10-d8bf-499b-82ba-e8f5e1becbd6 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:06.769768 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 118/222] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:06 2022] GET /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:06.771132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1d814-7179-4f08-bf3e-276dbe019b04 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:06.771630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1d814-7179-4f08-bf3e-276dbe019b04 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:06.772094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1d814-7179-4f08-bf3e-276dbe019b04 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:06.773656 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 119/223] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:06.779233 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1d814-7179-4f08-bf3e-276dbe019b04 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:06.805454 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:06.824845 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7566c8-af00-4c44-a218-b051abb496a6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:06.862138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870dcfc4-8c82-47bf-be01-08312bcfbb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:06.862655 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870dcfc4-8c82-47bf-be01-08312bcfbb2a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:06.862941 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870dcfc4-8c82-47bf-be01-08312bcfbb2a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:06.863266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870dcfc4-8c82-47bf-be01-08312bcfbb2a None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:06.864040 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870dcfc4-8c82-47bf-be01-08312bcfbb2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:06.864350 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870dcfc4-8c82-47bf-be01-08312bcfbb2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:06.868542 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870dcfc4-8c82-47bf-be01-08312bcfbb2a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:06.875478 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 105/224] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:06.876769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef9c93-5478-4289-80df-270caca9ae7e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:06.877063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef9c93-5478-4289-80df-270caca9ae7e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:06.877347 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef9c93-5478-4289-80df-270caca9ae7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:06.892768 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:06.894956 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1d814-7179-4f08-bf3e-276dbe019b04 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:06.896473 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-36ef9c93-5478-4289-80df-270caca9ae7e None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:06.897958 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-36ef9c93-5478-4289-80df-270caca9ae7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:06.935268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470b76b-1d3e-4c8b-8d8b-b9e336a0ef87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:06.936083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470b76b-1d3e-4c8b-8d8b-b9e336a0ef87 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:06.936507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470b76b-1d3e-4c8b-8d8b-b9e336a0ef87 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:06.936878 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470b76b-1d3e-4c8b-8d8b-b9e336a0ef87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:06.937821 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470b76b-1d3e-4c8b-8d8b-b9e336a0ef87 None admin] RBAC: Adding query filter params (role.id=8b9bae8f81854757b4f389d3bd22f595, scope.project.id=3233ec59a0994511b0032830b4ded6c2, user.id=5e42b1ec7f9f4871ab17c6f94ddd7f82) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:06.938237 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470b76b-1d3e-4c8b-8d8b-b9e336a0ef87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:06.939981 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470b76b-1d3e-4c8b-8d8b-b9e336a0ef87 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:06.946484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 120/225] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:53:06 2022] GET /identity/v3/role_assignments?user.id=5e42b1ec7f9f4871ab17c6f94ddd7f82&scope.project.id=3233ec59a0994511b0032830b4ded6c2&role.id=8b9bae8f81854757b4f389d3bd22f595 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:06.947577 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470b76b-1d3e-4c8b-8d8b-b9e336a0ef87 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:06.986737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 106/226] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:07.031641 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef9c93-5478-4289-80df-270caca9ae7e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:07.126072 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:07.138457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470b76b-1d3e-4c8b-8d8b-b9e336a0ef87 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:07.172989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979359c-53c7-4269-95db-c856d3198825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:07.174172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979359c-53c7-4269-95db-c856d3198825 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:07.174454 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979359c-53c7-4269-95db-c856d3198825 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:07.174725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979359c-53c7-4269-95db-c856d3198825 None admin] PATH_INFO: `/v3/users/6525e3c87b6c4fe783339d0d3765d462` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:07.175998 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e979359c-53c7-4269-95db-c856d3198825 None admin] RBAC: Authorizing `identity:get_user(user_id=6525e3c87b6c4fe783339d0d3765d462)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:07.177651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e979359c-53c7-4269-95db-c856d3198825 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:07.179463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 121/227] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:06 2022] GET /identity/v3/users/6525e3c87b6c4fe783339d0d3765d462 => generated 283 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:07.184992 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979359c-53c7-4269-95db-c856d3198825 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:07.304313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979359c-53c7-4269-95db-c856d3198825 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:07.315177 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef9c93-5478-4289-80df-270caca9ae7e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:07.339447 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:07.370377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef8b2d-3283-414f-a971-f1e1396a3459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:07.371252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef8b2d-3283-414f-a971-f1e1396a3459 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:07.371530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef8b2d-3283-414f-a971-f1e1396a3459 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:07.371802 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef8b2d-3283-414f-a971-f1e1396a3459 None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:07.373448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef8b2d-3283-414f-a971-f1e1396a3459 None admin] RBAC: Authorizing `identity:get_project(project_id=3233ec59a0994511b0032830b4ded6c2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:07.374913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef8b2d-3283-414f-a971-f1e1396a3459 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:07.376864 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 122/228] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:07 2022] GET /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:07.380243 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79274e26-8113-47ae-944d-2a18e7e8b6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:07.381161 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79274e26-8113-47ae-944d-2a18e7e8b6ae None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:07.381486 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79274e26-8113-47ae-944d-2a18e7e8b6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:07.381760 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79274e26-8113-47ae-944d-2a18e7e8b6ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:07.383326 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef8b2d-3283-414f-a971-f1e1396a3459 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:07.407406 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79274e26-8113-47ae-944d-2a18e7e8b6ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:07.421803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-79274e26-8113-47ae-944d-2a18e7e8b6ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:07.421803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:07.421803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:07.421803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:07.421803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:07.421803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:07.421803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:07.421803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:07.421803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:07.421803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:07.421803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:07.421803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:07.421803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:07.421803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:07.421803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:07.421803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:07.422705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:07.422705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:07.422705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:07.422705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:07.422705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:07.422705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:07.422705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:07.422705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:07.422705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:07.422705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:07.422705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:07.422705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:07.422705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:07.422705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:07.422705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:07.422705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:07.423485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:07.423485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:07.423485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:07.423485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:07.423485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:07.423485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:07.423485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:07.423485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:07.423485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:07.423485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:07.423485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:07.423485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:07.426196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 107/229] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:53:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:07.440763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:07.441806 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79274e26-8113-47ae-944d-2a18e7e8b6ae None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:07.571769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79274e26-8113-47ae-944d-2a18e7e8b6ae None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:07.592439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef8b2d-3283-414f-a971-f1e1396a3459 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:07.621721 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daef51a-0498-4e2c-85ef-5d6213c31d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:07.622519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daef51a-0498-4e2c-85ef-5d6213c31d4f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:07.622803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daef51a-0498-4e2c-85ef-5d6213c31d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:07.623072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daef51a-0498-4e2c-85ef-5d6213c31d4f None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:07.623829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daef51a-0498-4e2c-85ef-5d6213c31d4f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:07.624141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daef51a-0498-4e2c-85ef-5d6213c31d4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:07.625729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daef51a-0498-4e2c-85ef-5d6213c31d4f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:07.634656 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 108/230] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:07.640288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daef51a-0498-4e2c-85ef-5d6213c31d4f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:07.647009 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5526dc6-a638-405e-ab9e-a636da5c52af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:07.648533 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5526dc6-a638-405e-ab9e-a636da5c52af None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:07.649037 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5526dc6-a638-405e-ab9e-a636da5c52af None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:07.649503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5526dc6-a638-405e-ab9e-a636da5c52af None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2/users/6525e3c87b6c4fe783339d0d3765d462/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:07.657842 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5526dc6-a638-405e-ab9e-a636da5c52af None admin] RBAC: Authorizing `identity:create_grant(project_id=3233ec59a0994511b0032830b4ded6c2, user_id=6525e3c87b6c4fe783339d0d3765d462, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:07.659575 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5526dc6-a638-405e-ab9e-a636da5c52af None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:07.677169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:07.681172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 123/231] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:53:07 2022] PUT /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2/users/6525e3c87b6c4fe783339d0d3765d462/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:53:07.774185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daef51a-0498-4e2c-85ef-5d6213c31d4f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:07.830068 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d1b3c-3cd8-4237-8cbb-2a9bb39ad2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:07.830962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d1b3c-3cd8-4237-8cbb-2a9bb39ad2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:07.831242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d1b3c-3cd8-4237-8cbb-2a9bb39ad2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:07.831505 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d1b3c-3cd8-4237-8cbb-2a9bb39ad2a2 None admin] PATH_INFO: `/v3/users/6525e3c87b6c4fe783339d0d3765d462` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:07.839491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d1b3c-3cd8-4237-8cbb-2a9bb39ad2a2 None admin] RBAC: Authorizing `identity:get_user(user_id=6525e3c87b6c4fe783339d0d3765d462)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:07.839491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d1b3c-3cd8-4237-8cbb-2a9bb39ad2a2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:07.840955 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 109/232] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:07 2022] GET /identity/v3/users/6525e3c87b6c4fe783339d0d3765d462 => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:07.850486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5526dc6-a638-405e-ab9e-a636da5c52af None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:07.942821 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:07.963195 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5526dc6-a638-405e-ab9e-a636da5c52af None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:08.000738 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900f460-9013-4755-8cce-aca8f005daae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:08.001602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900f460-9013-4755-8cce-aca8f005daae None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:08.001878 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900f460-9013-4755-8cce-aca8f005daae None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:08.002151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900f460-9013-4755-8cce-aca8f005daae None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:08.007831 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900f460-9013-4755-8cce-aca8f005daae None admin] RBAC: Authorizing `identity:get_project(project_id=3233ec59a0994511b0032830b4ded6c2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:08.009428 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900f460-9013-4755-8cce-aca8f005daae None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:08.011740 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 124/233] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:07 2022] GET /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:08.017536 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d1b3c-3cd8-4237-8cbb-2a9bb39ad2a2 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:08.123642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d1b3c-3cd8-4237-8cbb-2a9bb39ad2a2 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:08.126051 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:08.157809 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f2a2f-5012-4812-8667-44f0b9a3c00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:08.158697 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f2a2f-5012-4812-8667-44f0b9a3c00a None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:08.159015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f2a2f-5012-4812-8667-44f0b9a3c00a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:08.159285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f2a2f-5012-4812-8667-44f0b9a3c00a None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2/users/6525e3c87b6c4fe783339d0d3765d462/roles/6322fb4a438d4a389c43d05248665e72` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:08.161543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f2a2f-5012-4812-8667-44f0b9a3c00a None admin] RBAC: Authorizing `identity:create_grant(project_id=3233ec59a0994511b0032830b4ded6c2, user_id=6525e3c87b6c4fe783339d0d3765d462, role_id=6322fb4a438d4a389c43d05248665e72)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:08.163087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f2a2f-5012-4812-8667-44f0b9a3c00a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:08.177728 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 110/234] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:53:08 2022] PUT /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2/users/6525e3c87b6c4fe783339d0d3765d462/roles/6322fb4a438d4a389c43d05248665e72 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:53:08.691121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e7436-8652-44ea-8486-2b88061caf4e None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:08.691633 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e7436-8652-44ea-8486-2b88061caf4e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:08.691707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e7436-8652-44ea-8486-2b88061caf4e None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:08.693352 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 125/235] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:08.699076 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da072c7-2635-4f90-bfb2-7019a692b727 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:08.699523 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da072c7-2635-4f90-bfb2-7019a692b727 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:08.699626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da072c7-2635-4f90-bfb2-7019a692b727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:08.716953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2da072c7-2635-4f90-bfb2-7019a692b727 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:08.718136 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2da072c7-2635-4f90-bfb2-7019a692b727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:08.761804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551786c-fe0b-4346-882d-52c86f5f823b None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:08.762110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551786c-fe0b-4346-882d-52c86f5f823b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:08.762414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551786c-fe0b-4346-882d-52c86f5f823b None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:08.763976 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 126/236] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:53:08.777839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbf27d-0add-486c-bf96-745a1bbed000 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:08.778111 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbf27d-0add-486c-bf96-745a1bbed000 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:08.778378 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbf27d-0add-486c-bf96-745a1bbed000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:08.804755 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5ffbf27d-0add-486c-bf96-745a1bbed000 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:08.805941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5ffbf27d-0add-486c-bf96-745a1bbed000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:08.817604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 111/237] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:53:08.891794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da072c7-2635-4f90-bfb2-7019a692b727 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:08.913750 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 127/238] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:53:08.950543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbf27d-0add-486c-bf96-745a1bbed000 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:08.990271 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:08.996289 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da072c7-2635-4f90-bfb2-7019a692b727 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:09.035143 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f0f48-0403-43fd-a8e7-62093e4766e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:09.035943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f0f48-0403-43fd-a8e7-62093e4766e7 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:09.036219 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f0f48-0403-43fd-a8e7-62093e4766e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:09.036522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f0f48-0403-43fd-a8e7-62093e4766e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:09.037716 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f0f48-0403-43fd-a8e7-62093e4766e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:09.039619 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f0f48-0403-43fd-a8e7-62093e4766e7 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:09.041377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 112/239] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:09.047084 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f0f48-0403-43fd-a8e7-62093e4766e7 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:09.112918 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:09.145707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbf27d-0add-486c-bf96-745a1bbed000 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:09.146894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f0f48-0403-43fd-a8e7-62093e4766e7 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:09.183004 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47980233-c756-4541-851a-fdc6507fb128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:09.183679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47980233-c756-4541-851a-fdc6507fb128 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:09.183955 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47980233-c756-4541-851a-fdc6507fb128 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:09.184225 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47980233-c756-4541-851a-fdc6507fb128 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:09.184969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47980233-c756-4541-851a-fdc6507fb128 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:09.188430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47980233-c756-4541-851a-fdc6507fb128 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:09.219177 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 113/240] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Jun 25 15:53:09 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 15:53:09.225997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b606f4-f805-4b6a-a4cb-2716aa82d39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:09.227267 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b606f4-f805-4b6a-a4cb-2716aa82d39e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:09.227744 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b606f4-f805-4b6a-a4cb-2716aa82d39e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:09.228210 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b606f4-f805-4b6a-a4cb-2716aa82d39e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:09.247112 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b606f4-f805-4b6a-a4cb-2716aa82d39e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:09.255431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-55b606f4-f805-4b6a-a4cb-2716aa82d39e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 25 15:53:09.255431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:09.255431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:09.255431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:09.255431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:09.255431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:09.255431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:09.255431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:09.255431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:09.255431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:09.255431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:09.255431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:09.255431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:09.255431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:09.255431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:09.255431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:09.256370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:09.256370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:09.256370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:09.256370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:09.256370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:09.256370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:09.256370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:09.256370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:09.256370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:09.256370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:09.256370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:09.256370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:09.256370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:09.256370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:09.256370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:09.256370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:09.262327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:09.262327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:09.262327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:09.262327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:09.262327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:09.262327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:09.262327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:09.262327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:09.262327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:09.262327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:09.262327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 25 15:53:09.262327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:09.265108 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 128/241] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:09.271014 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47980233-c756-4541-851a-fdc6507fb128 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:09.298021 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:09.413733 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47980233-c756-4541-851a-fdc6507fb128 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:09.426033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380f0aa-8f80-45cd-98e5-39f7a51a12f1 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:09.426652 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380f0aa-8f80-45cd-98e5-39f7a51a12f1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:09.427125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380f0aa-8f80-45cd-98e5-39f7a51a12f1 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:09.428880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 129/242] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:09.456921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4ea91-b453-4ddf-b8fe-a9b6fd0879c0 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:09.457527 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4ea91-b453-4ddf-b8fe-a9b6fd0879c0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:09.458000 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4ea91-b453-4ddf-b8fe-a9b6fd0879c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:09.466695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8abcb9-4b08-47ed-8e23-644dc9f698d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:09.467415 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8abcb9-4b08-47ed-8e23-644dc9f698d0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:09.467700 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8abcb9-4b08-47ed-8e23-644dc9f698d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:09.467969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8abcb9-4b08-47ed-8e23-644dc9f698d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:09.470137 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8abcb9-4b08-47ed-8e23-644dc9f698d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:09.470394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8abcb9-4b08-47ed-8e23-644dc9f698d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:09.471790 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8abcb9-4b08-47ed-8e23-644dc9f698d0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:09.480811 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 114/243] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:09.488436 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-84a4ea91-b453-4ddf-b8fe-a9b6fd0879c0 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:09.488436 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-84a4ea91-b453-4ddf-b8fe-a9b6fd0879c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:09.489577 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8abcb9-4b08-47ed-8e23-644dc9f698d0 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:09.577869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 130/244] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:53:09.587666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:09.609534 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4ea91-b453-4ddf-b8fe-a9b6fd0879c0 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:09.618247 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8abcb9-4b08-47ed-8e23-644dc9f698d0 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:09.669566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbdaec-f422-4384-a52e-d48fa2ee98b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:09.670742 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbdaec-f422-4384-a52e-d48fa2ee98b6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:09.671028 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbdaec-f422-4384-a52e-d48fa2ee98b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:09.671306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbdaec-f422-4384-a52e-d48fa2ee98b6 None admin] PATH_INFO: `/v3/users/21b25f80c7d2495685cceff558ae23d8` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:09.689232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbdaec-f422-4384-a52e-d48fa2ee98b6 None admin] RBAC: Authorizing `identity:get_user(user_id=21b25f80c7d2495685cceff558ae23d8)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:09.691009 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbdaec-f422-4384-a52e-d48fa2ee98b6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:09.692458 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:09.694074 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 115/245] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:09 2022] GET /identity/v3/users/21b25f80c7d2495685cceff558ae23d8 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:09.699925 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbdaec-f422-4384-a52e-d48fa2ee98b6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:09.732105 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4ea91-b453-4ddf-b8fe-a9b6fd0879c0 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:09.775933 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d70c0a-6369-422c-95b8-fa10bd99ec78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:09.777026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d70c0a-6369-422c-95b8-fa10bd99ec78 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:09.777464 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d70c0a-6369-422c-95b8-fa10bd99ec78 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:09.777868 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d70c0a-6369-422c-95b8-fa10bd99ec78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:09.789536 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d70c0a-6369-422c-95b8-fa10bd99ec78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:09.798517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-f0d70c0a-6369-422c-95b8-fa10bd99ec78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:09.798517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:09.798517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:09.798517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:09.798517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:09.798517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:09.798517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:09.798517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:09.798517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:09.798517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:09.798517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:09.798517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:09.798517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:09.798517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:09.798517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:09.798517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:09.799606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:09.799606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:09.799606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:09.799606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:09.799606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:09.799606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:09.799606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:09.799606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:09.799606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:09.799606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:09.799606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:09.799606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:09.799606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:09.799606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:09.799606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:09.799606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:09.800585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:09.800585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:09.800585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:09.800585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:09.800585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:09.800585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:09.800585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:09.800585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:09.800585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:09.800585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:09.800585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:09.800585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:09.802933 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 131/246] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:09.810589 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d70c0a-6369-422c-95b8-fa10bd99ec78 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:09.830509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbdaec-f422-4384-a52e-d48fa2ee98b6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:09.884303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312e9b5-a0f3-4013-99f9-f6558b19d25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:09.885577 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312e9b5-a0f3-4013-99f9-f6558b19d25d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:09.886109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312e9b5-a0f3-4013-99f9-f6558b19d25d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:09.886832 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312e9b5-a0f3-4013-99f9-f6558b19d25d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:09.898748 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:09.962981 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d70c0a-6369-422c-95b8-fa10bd99ec78 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:09.986249 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312e9b5-a0f3-4013-99f9-f6558b19d25d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:09.988043 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312e9b5-a0f3-4013-99f9-f6558b19d25d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:10.004064 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:10.012752 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b83bf8-3a09-4f81-8ef3-fae8ef595687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:10.013647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b83bf8-3a09-4f81-8ef3-fae8ef595687 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:10.014183 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b83bf8-3a09-4f81-8ef3-fae8ef595687 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:10.014623 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b83bf8-3a09-4f81-8ef3-fae8ef595687 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:10.015807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b83bf8-3a09-4f81-8ef3-fae8ef595687 None admin] RBAC: Adding query filter params (name=member) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:10.016257 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b83bf8-3a09-4f81-8ef3-fae8ef595687 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:10.018125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b83bf8-3a09-4f81-8ef3-fae8ef595687 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:10.030378 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 132/247] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:10.036834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b83bf8-3a09-4f81-8ef3-fae8ef595687 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:10.128286 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 116/248] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:10.141758 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312e9b5-a0f3-4013-99f9-f6558b19d25d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:10.196260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b83bf8-3a09-4f81-8ef3-fae8ef595687 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:10.239733 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:10.272364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342aab72-152f-471d-9ec2-0d9589e6447a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:10.273288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342aab72-152f-471d-9ec2-0d9589e6447a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:10.273577 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342aab72-152f-471d-9ec2-0d9589e6447a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:10.273855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342aab72-152f-471d-9ec2-0d9589e6447a None admin] PATH_INFO: `/v3/users/5e42b1ec7f9f4871ab17c6f94ddd7f82` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:10.275131 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342aab72-152f-471d-9ec2-0d9589e6447a None admin] RBAC: Authorizing `identity:get_user(user_id=5e42b1ec7f9f4871ab17c6f94ddd7f82)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:10.276638 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342aab72-152f-471d-9ec2-0d9589e6447a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:10.278530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 133/249] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:10 2022] GET /identity/v3/users/5e42b1ec7f9f4871ab17c6f94ddd7f82 => generated 297 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:10.283641 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342aab72-152f-471d-9ec2-0d9589e6447a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:10.370788 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312e9b5-a0f3-4013-99f9-f6558b19d25d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:10.400795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:10.457936 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342aab72-152f-471d-9ec2-0d9589e6447a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:10.460638 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8f62a-898e-4ffb-a916-c42366d97cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:10.461525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8f62a-898e-4ffb-a916-c42366d97cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:10.461803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8f62a-898e-4ffb-a916-c42366d97cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:10.462076 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8f62a-898e-4ffb-a916-c42366d97cd0 None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:10.465172 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8f62a-898e-4ffb-a916-c42366d97cd0 None admin] RBAC: Authorizing `identity:get_project(project_id=3233ec59a0994511b0032830b4ded6c2)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:10.466672 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8f62a-898e-4ffb-a916-c42366d97cd0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:10.468460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 117/250] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:10 2022] GET /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2 => generated 302 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:10.469691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e261699-6220-45d8-9825-7911f902e56a None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:10.469977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e261699-6220-45d8-9825-7911f902e56a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:10.470256 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e261699-6220-45d8-9825-7911f902e56a None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:10.471487 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 118/251] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:10.483728 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e261699-6220-45d8-9825-7911f902e56a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:10.536863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc5d73-bf1e-4034-9fb9-aecdd89629d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:10.537981 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc5d73-bf1e-4034-9fb9-aecdd89629d0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:10.538400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc5d73-bf1e-4034-9fb9-aecdd89629d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:10.538825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc5d73-bf1e-4034-9fb9-aecdd89629d0 None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:10.545164 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc5d73-bf1e-4034-9fb9-aecdd89629d0 None admin] RBAC: Authorizing `identity:get_project(project_id=3233ec59a0994511b0032830b4ded6c2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:10.546812 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc5d73-bf1e-4034-9fb9-aecdd89629d0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:10.549095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 134/252] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:10 2022] GET /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:10.550400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0958bc5-9fbe-47df-a2f9-16c47d0040ea None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:10.550841 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0958bc5-9fbe-47df-a2f9-16c47d0040ea None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:10.551249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0958bc5-9fbe-47df-a2f9-16c47d0040ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:10.593496 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f0958bc5-9fbe-47df-a2f9-16c47d0040ea None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:10.594934 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f0958bc5-9fbe-47df-a2f9-16c47d0040ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:10.619860 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e261699-6220-45d8-9825-7911f902e56a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:10.628517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:10.661057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56421d-738d-459a-9877-d8b4269d3b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:10.661863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56421d-738d-459a-9877-d8b4269d3b8d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:10.662149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56421d-738d-459a-9877-d8b4269d3b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:10.662429 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56421d-738d-459a-9877-d8b4269d3b8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:10.663440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56421d-738d-459a-9877-d8b4269d3b8d None admin] RBAC: Adding query filter params (role.id=f70856a334f7479fb00bac40f893e9f0, scope.project.id=3233ec59a0994511b0032830b4ded6c2, user.id=21b25f80c7d2495685cceff558ae23d8) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:10.663760 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56421d-738d-459a-9877-d8b4269d3b8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:10.665630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56421d-738d-459a-9877-d8b4269d3b8d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:10.672428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 119/253] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:53:10 2022] GET /identity/v3/role_assignments?user.id=21b25f80c7d2495685cceff558ae23d8&scope.project.id=3233ec59a0994511b0032830b4ded6c2&role.id=f70856a334f7479fb00bac40f893e9f0 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:10.673469 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56421d-738d-459a-9877-d8b4269d3b8d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:10.793178 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56421d-738d-459a-9877-d8b4269d3b8d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:10.798184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 135/254] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:10.848004 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c7996-9cc6-42ab-b39e-46f67839acef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:10.848948 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c7996-9cc6-42ab-b39e-46f67839acef None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:10.849232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c7996-9cc6-42ab-b39e-46f67839acef None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:10.849509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c7996-9cc6-42ab-b39e-46f67839acef None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2/users/5e42b1ec7f9f4871ab17c6f94ddd7f82/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:10.851664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c7996-9cc6-42ab-b39e-46f67839acef None admin] RBAC: Authorizing `identity:create_grant(project_id=3233ec59a0994511b0032830b4ded6c2, user_id=5e42b1ec7f9f4871ab17c6f94ddd7f82, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:10.853278 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c7996-9cc6-42ab-b39e-46f67839acef None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:10.868079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0958bc5-9fbe-47df-a2f9-16c47d0040ea None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:10.879937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 120/255] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:53:10 2022] PUT /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2/users/5e42b1ec7f9f4871ab17c6f94ddd7f82/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:53:10.944824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:10.992201 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0958bc5-9fbe-47df-a2f9-16c47d0040ea None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:11.023687 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7f74c-0205-4eac-b4be-e277e92a91fa None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:11.024610 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7f74c-0205-4eac-b4be-e277e92a91fa None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:11.025122 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7f74c-0205-4eac-b4be-e277e92a91fa None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:11.027620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 121/256] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:53:11.031457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89720bc5-cefd-46fc-a398-5350ff9ae0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:11.032542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89720bc5-cefd-46fc-a398-5350ff9ae0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:11.033029 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89720bc5-cefd-46fc-a398-5350ff9ae0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:11.033545 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89720bc5-cefd-46fc-a398-5350ff9ae0d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:11.035429 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd478257-d052-417c-a234-4ac3e7baba1f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:11.035955 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd478257-d052-417c-a234-4ac3e7baba1f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:11.036440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd478257-d052-417c-a234-4ac3e7baba1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:11.058610 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89720bc5-cefd-46fc-a398-5350ff9ae0d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:11.065349 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-fd478257-d052-417c-a234-4ac3e7baba1f None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:11.069356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-fd478257-d052-417c-a234-4ac3e7baba1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:11.073107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-89720bc5-cefd-46fc-a398-5350ff9ae0d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:53:11.073107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:11.073107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:11.073107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:11.073107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:11.073107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:11.073107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:11.073107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:11.073107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:11.073107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:11.073107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:11.073107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:11.073107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:11.073107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:11.073107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:11.073107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:11.073996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:11.073996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:11.073996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:11.073996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:11.073996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:11.073996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:11.073996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:11.073996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:11.073996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:11.073996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:11.073996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:11.073996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:11.073996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:11.073996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:11.073996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:11.073996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:11.074834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:11.074834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:11.074834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:11.074834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:11.074834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:11.074834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:11.074834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:11.074834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:11.074834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:11.074834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:11.074834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:53:11.074834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:11.081132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 136/257] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:53:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:11.106154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89720bc5-cefd-46fc-a398-5350ff9ae0d2 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:11.125349 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:11.188065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 122/258] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:11.217526 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd478257-d052-417c-a234-4ac3e7baba1f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:11.271537 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89720bc5-cefd-46fc-a398-5350ff9ae0d2 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:11.291933 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:11.322048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d011f1f-76f8-4337-86ac-1cb355100730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:11.322669 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d011f1f-76f8-4337-86ac-1cb355100730 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:11.322947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d011f1f-76f8-4337-86ac-1cb355100730 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:11.323219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d011f1f-76f8-4337-86ac-1cb355100730 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:11.324001 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d011f1f-76f8-4337-86ac-1cb355100730 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:11.324308 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d011f1f-76f8-4337-86ac-1cb355100730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:11.325782 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d011f1f-76f8-4337-86ac-1cb355100730 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:11.344726 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 137/259] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:11.350381 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d011f1f-76f8-4337-86ac-1cb355100730 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:11.392098 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd478257-d052-417c-a234-4ac3e7baba1f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:11.449136 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfcc19-2aaf-4828-a9ea-cd1fb4c44ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:11.449971 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfcc19-2aaf-4828-a9ea-cd1fb4c44ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:11.450256 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfcc19-2aaf-4828-a9ea-cd1fb4c44ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:11.450533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfcc19-2aaf-4828-a9ea-cd1fb4c44ce8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:11.465896 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cfcc19-2aaf-4828-a9ea-cd1fb4c44ce8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:11.482329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-31cfcc19-2aaf-4828-a9ea-cd1fb4c44ce8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:11.482329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:11.482329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:11.482329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:11.482329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:11.482329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:11.482329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:11.482329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:11.482329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:11.482329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:11.482329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:11.482329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:11.482329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:11.482329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:11.482329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:11.482329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:11.483266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:11.483266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:11.483266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:11.483266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:11.483266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:11.483266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:11.483266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:11.483266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:11.483266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:11.483266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:11.483266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:11.483266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:11.483266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:11.483266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:11.483266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:11.483266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:11.484064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:11.484064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:11.484064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:11.484064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:11.484064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:11.484064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:11.484064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:11.484064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:11.484064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:11.484064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:11.484064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:11.484064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:11.484064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 123/260] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:53:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:11.490837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfcc19-2aaf-4828-a9ea-cd1fb4c44ce8 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:11.531579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:11.549287 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d011f1f-76f8-4337-86ac-1cb355100730 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:11.611084 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fdcfa-5460-420a-be29-ba93ad04db5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:11.612006 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fdcfa-5460-420a-be29-ba93ad04db5d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:11.612290 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fdcfa-5460-420a-be29-ba93ad04db5d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:11.612594 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fdcfa-5460-420a-be29-ba93ad04db5d None admin] PATH_INFO: `/v3/users/6525e3c87b6c4fe783339d0d3765d462` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:11.617130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fdcfa-5460-420a-be29-ba93ad04db5d None admin] RBAC: Authorizing `identity:get_user(user_id=6525e3c87b6c4fe783339d0d3765d462)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:11.618702 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fdcfa-5460-420a-be29-ba93ad04db5d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:11.620811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 138/261] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:11 2022] GET /identity/v3/users/6525e3c87b6c4fe783339d0d3765d462 => generated 283 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:11.626374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fdcfa-5460-420a-be29-ba93ad04db5d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:11.656242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfcc19-2aaf-4828-a9ea-cd1fb4c44ce8 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:11.697191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:11.713700 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9881a-1638-4564-99e9-381a3c053515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:11.714389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9881a-1638-4564-99e9-381a3c053515 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:11.714671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9881a-1638-4564-99e9-381a3c053515 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:11.714947 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9881a-1638-4564-99e9-381a3c053515 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:11.715713 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9881a-1638-4564-99e9-381a3c053515 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:11.716019 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9881a-1638-4564-99e9-381a3c053515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:11.717489 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9881a-1638-4564-99e9-381a3c053515 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:11.732236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 124/262] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:11.737645 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9881a-1638-4564-99e9-381a3c053515 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:11.752361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fdcfa-5460-420a-be29-ba93ad04db5d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:11.791340 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938e3de-718e-4206-be0a-6059154333d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:11.792184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938e3de-718e-4206-be0a-6059154333d8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:11.792496 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938e3de-718e-4206-be0a-6059154333d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:11.792785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938e3de-718e-4206-be0a-6059154333d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:11.870271 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f938e3de-718e-4206-be0a-6059154333d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:11.871251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f938e3de-718e-4206-be0a-6059154333d8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:11.928771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:11.930316 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:11.991280 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9881a-1638-4564-99e9-381a3c053515 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:11.993314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 139/263] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 25 15:53:12.002375 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938e3de-718e-4206-be0a-6059154333d8 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:12.061233 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40225a33-def9-467a-a89d-8b61a4c4b55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:12.062153 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40225a33-def9-467a-a89d-8b61a4c4b55e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:12.062440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40225a33-def9-467a-a89d-8b61a4c4b55e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:12.062722 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40225a33-def9-467a-a89d-8b61a4c4b55e None admin] PATH_INFO: `/v3/users/6525e3c87b6c4fe783339d0d3765d462` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:12.063933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40225a33-def9-467a-a89d-8b61a4c4b55e None admin] RBAC: Authorizing `identity:get_user(user_id=6525e3c87b6c4fe783339d0d3765d462)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:12.065462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40225a33-def9-467a-a89d-8b61a4c4b55e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:12.067398 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 125/264] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:11 2022] GET /identity/v3/users/6525e3c87b6c4fe783339d0d3765d462 => generated 283 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:12.073428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40225a33-def9-467a-a89d-8b61a4c4b55e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:12.182843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40225a33-def9-467a-a89d-8b61a4c4b55e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:12.219321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961ff85-f9bf-4b90-82a7-21593aae1857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:12.219982 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961ff85-f9bf-4b90-82a7-21593aae1857 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:12.220265 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961ff85-f9bf-4b90-82a7-21593aae1857 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:12.220568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961ff85-f9bf-4b90-82a7-21593aae1857 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:12.241427 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:12.274459 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938e3de-718e-4206-be0a-6059154333d8 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:12.297270 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2961ff85-f9bf-4b90-82a7-21593aae1857 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:12.298759 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2961ff85-f9bf-4b90-82a7-21593aae1857 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:12.385724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1674e-8405-4f55-9c72-406c818ccca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:12.386615 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1674e-8405-4f55-9c72-406c818ccca7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:12.386899 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1674e-8405-4f55-9c72-406c818ccca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:12.387169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1674e-8405-4f55-9c72-406c818ccca7 None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:12.388392 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1674e-8405-4f55-9c72-406c818ccca7 None admin] RBAC: Authorizing `identity:get_project(project_id=3233ec59a0994511b0032830b4ded6c2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:12.389868 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1674e-8405-4f55-9c72-406c818ccca7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:12.395278 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 140/265] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:12 2022] GET /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2 => generated 302 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:12.396636 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73a8b8-0b5e-444b-a7f2-97592f5def77 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:12.396921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73a8b8-0b5e-444b-a7f2-97592f5def77 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:12.397197 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73a8b8-0b5e-444b-a7f2-97592f5def77 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:12.403048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 141/266] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:12.403878 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73a8b8-0b5e-444b-a7f2-97592f5def77 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:12.414751 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 126/267] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 25 15:53:12.421382 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4eabe-d834-4527-9a03-d9f9aaa0511c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:12.421678 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4eabe-d834-4527-9a03-d9f9aaa0511c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:12.421967 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4eabe-d834-4527-9a03-d9f9aaa0511c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:12.458523 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e0d4eabe-d834-4527-9a03-d9f9aaa0511c None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:12.459658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e0d4eabe-d834-4527-9a03-d9f9aaa0511c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:12.460767 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:12.572550 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:12.615441 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 127/268] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:12.616323 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4eabe-d834-4527-9a03-d9f9aaa0511c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:12.629612 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73a8b8-0b5e-444b-a7f2-97592f5def77 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:12.679806 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa542b66-55b2-46f8-a8be-2838bf47fd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:12.680502 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa542b66-55b2-46f8-a8be-2838bf47fd3b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:12.680785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa542b66-55b2-46f8-a8be-2838bf47fd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:12.681064 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa542b66-55b2-46f8-a8be-2838bf47fd3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:12.682592 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa542b66-55b2-46f8-a8be-2838bf47fd3b None admin] RBAC: Adding query filter params (role.id=b1e13c449fa64b3e88b6832bb767214b, scope.project.id=3233ec59a0994511b0032830b4ded6c2, user.id=6525e3c87b6c4fe783339d0d3765d462) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:12.682908 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa542b66-55b2-46f8-a8be-2838bf47fd3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:12.684929 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa542b66-55b2-46f8-a8be-2838bf47fd3b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:12.693609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 142/269] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:53:12 2022] GET /identity/v3/role_assignments?user.id=6525e3c87b6c4fe783339d0d3765d462&scope.project.id=3233ec59a0994511b0032830b4ded6c2&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:12.694685 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cc1f9-59ee-47b6-bb17-d9305f52746c None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:12.694975 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cc1f9-59ee-47b6-bb17-d9305f52746c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:12.695260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cc1f9-59ee-47b6-bb17-d9305f52746c None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:12.700652 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 143/270] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:12.701453 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cc1f9-59ee-47b6-bb17-d9305f52746c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:12.738244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4eabe-d834-4527-9a03-d9f9aaa0511c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:12.779961 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48544ee-b591-482c-8d71-8b99e1ca071d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:12.780882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48544ee-b591-482c-8d71-8b99e1ca071d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:12.781165 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48544ee-b591-482c-8d71-8b99e1ca071d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:12.781829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48544ee-b591-482c-8d71-8b99e1ca071d None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:12.783307 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48544ee-b591-482c-8d71-8b99e1ca071d None admin] RBAC: Authorizing `identity:get_project(project_id=3233ec59a0994511b0032830b4ded6c2)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:12.784854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48544ee-b591-482c-8d71-8b99e1ca071d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:12.786991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 128/271] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:12 2022] GET /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:12.788971 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac02f38-c3af-4b7c-88d5-7ae7897a9660 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:12.789310 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac02f38-c3af-4b7c-88d5-7ae7897a9660 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:12.789774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac02f38-c3af-4b7c-88d5-7ae7897a9660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:12.809776 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2ac02f38-c3af-4b7c-88d5-7ae7897a9660 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:12.811305 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2ac02f38-c3af-4b7c-88d5-7ae7897a9660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:12.908978 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:12.918331 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:12.949147 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cc1f9-59ee-47b6-bb17-d9305f52746c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:12.954797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 129/272] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:12.955663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac02f38-c3af-4b7c-88d5-7ae7897a9660 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:13.020215 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbf885-517d-47ee-9019-e611244dd5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:13.021150 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbf885-517d-47ee-9019-e611244dd5fc None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:13.021437 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbf885-517d-47ee-9019-e611244dd5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:13.021716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbf885-517d-47ee-9019-e611244dd5fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:13.023719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbf885-517d-47ee-9019-e611244dd5fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:13.025977 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbf885-517d-47ee-9019-e611244dd5fc None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:13.027519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 144/273] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:13.028297 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbf885-517d-47ee-9019-e611244dd5fc None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:13.078444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac02f38-c3af-4b7c-88d5-7ae7897a9660 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:13.118631 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6010f-039f-4cd5-990a-c84089b8a7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:13.120572 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6010f-039f-4cd5-990a-c84089b8a7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:13.121066 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6010f-039f-4cd5-990a-c84089b8a7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:13.121555 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6010f-039f-4cd5-990a-c84089b8a7e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:13.122607 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6010f-039f-4cd5-990a-c84089b8a7e2 None admin] RBAC: Adding query filter params (role.id=6322fb4a438d4a389c43d05248665e72, scope.project.id=3233ec59a0994511b0032830b4ded6c2, user.id=6525e3c87b6c4fe783339d0d3765d462) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:13.123231 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6010f-039f-4cd5-990a-c84089b8a7e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:13.125172 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6010f-039f-4cd5-990a-c84089b8a7e2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:13.135633 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 130/274] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:53:12 2022] GET /identity/v3/role_assignments?user.id=6525e3c87b6c4fe783339d0d3765d462&scope.project.id=3233ec59a0994511b0032830b4ded6c2&role.id=6322fb4a438d4a389c43d05248665e72 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:13.136878 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6010f-039f-4cd5-990a-c84089b8a7e2 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:13.140599 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbf885-517d-47ee-9019-e611244dd5fc None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:13.185160 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f57ee-13cf-4914-9147-c1a09e27c0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:13.186391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f57ee-13cf-4914-9147-c1a09e27c0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:13.186940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f57ee-13cf-4914-9147-c1a09e27c0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:13.187489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f57ee-13cf-4914-9147-c1a09e27c0e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:13.195042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:13.209305 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f57ee-13cf-4914-9147-c1a09e27c0e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:13.219729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-392f57ee-13cf-4914-9147-c1a09e27c0e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:13.219729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:13.219729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:13.219729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:13.219729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:13.219729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:13.219729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:13.219729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:13.219729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:13.219729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:13.219729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:13.219729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:13.219729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:13.219729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:13.219729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:13.219729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:13.220707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:13.220707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:13.220707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:13.220707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:13.220707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:13.220707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:13.220707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:13.220707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:13.220707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:13.220707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:13.220707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:13.220707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:13.220707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:13.220707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:13.220707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:13.220707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:13.221508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:13.221508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:13.221508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:13.221508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:13.221508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:13.221508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:13.221508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:13.221508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:13.221508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:13.221508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:13.221508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:13.221508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:13.224260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 145/275] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:13.236440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f57ee-13cf-4914-9147-c1a09e27c0e9 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:13.267467 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:13.316352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6010f-039f-4cd5-990a-c84089b8a7e2 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:13.372276 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc85a9c-3a52-4dba-99d5-9693dfdcaed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:13.372988 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc85a9c-3a52-4dba-99d5-9693dfdcaed2 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:13.373318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc85a9c-3a52-4dba-99d5-9693dfdcaed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:13.373600 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc85a9c-3a52-4dba-99d5-9693dfdcaed2 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:13.374320 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc85a9c-3a52-4dba-99d5-9693dfdcaed2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:13.375776 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc85a9c-3a52-4dba-99d5-9693dfdcaed2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:13.404282 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 131/276] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Jun 25 15:53:13 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:53:13.414482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f57ee-13cf-4914-9147-c1a09e27c0e9 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:13.458728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ed218-0958-4492-b75b-7b8c24602e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:13.459962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ed218-0958-4492-b75b-7b8c24602e5f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:13.460488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ed218-0958-4492-b75b-7b8c24602e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:13.461014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ed218-0958-4492-b75b-7b8c24602e5f None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:13.462067 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ed218-0958-4492-b75b-7b8c24602e5f None admin] RBAC: Adding query filter params (name=member) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:13.462665 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ed218-0958-4492-b75b-7b8c24602e5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:13.465284 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ed218-0958-4492-b75b-7b8c24602e5f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:13.476330 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 146/277] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:13.485616 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc85a9c-3a52-4dba-99d5-9693dfdcaed2 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:13.524188 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383aca8a-8f4a-44a5-a7a1-ffdfbd72a036 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:13.524560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383aca8a-8f4a-44a5-a7a1-ffdfbd72a036 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:13.524791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383aca8a-8f4a-44a5-a7a1-ffdfbd72a036 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:13.527884 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 147/278] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:13.532498 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc6aeb-f055-4d0d-bb83-8814a7a88b7a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:13.532786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc6aeb-f055-4d0d-bb83-8814a7a88b7a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:13.533061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc6aeb-f055-4d0d-bb83-8814a7a88b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:13.553929 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e3fc6aeb-f055-4d0d-bb83-8814a7a88b7a None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:13.555837 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e3fc6aeb-f055-4d0d-bb83-8814a7a88b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:13.615054 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:13.619102 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc85a9c-3a52-4dba-99d5-9693dfdcaed2 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:13.640575 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 148/279] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:53:13.657186 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1150c8-fd57-448c-8737-5778e5605bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:13.658084 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1150c8-fd57-448c-8737-5778e5605bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:13.658365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1150c8-fd57-448c-8737-5778e5605bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:13.658638 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1150c8-fd57-448c-8737-5778e5605bb6 None admin] PATH_INFO: `/v3/users/80e142e6def84c2e8be21c9a73583e45` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:13.663875 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc6aeb-f055-4d0d-bb83-8814a7a88b7a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:13.666437 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:13.675093 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1150c8-fd57-448c-8737-5778e5605bb6 None admin] RBAC: Authorizing `identity:get_user(user_id=80e142e6def84c2e8be21c9a73583e45)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:13.676642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1150c8-fd57-448c-8737-5778e5605bb6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:13.680507 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 132/280] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:13 2022] GET /identity/v3/users/80e142e6def84c2e8be21c9a73583e45 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:13.686360 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1150c8-fd57-448c-8737-5778e5605bb6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:13.796485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc6aeb-f055-4d0d-bb83-8814a7a88b7a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:13.799877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1150c8-fd57-448c-8737-5778e5605bb6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:13.837069 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c656788-99e3-4011-97e4-403d1f56c992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:13.838749 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c656788-99e3-4011-97e4-403d1f56c992 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:13.839104 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c656788-99e3-4011-97e4-403d1f56c992 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:13.839455 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c656788-99e3-4011-97e4-403d1f56c992 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:13.841796 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ecc75-89f6-4a83-82e7-08215a7fb371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:13.842682 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ecc75-89f6-4a83-82e7-08215a7fb371 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:13.843141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ecc75-89f6-4a83-82e7-08215a7fb371 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:13.843586 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ecc75-89f6-4a83-82e7-08215a7fb371 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:13.844640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ecc75-89f6-4a83-82e7-08215a7fb371 None admin] RBAC: Adding query filter params (role.id=8b9bae8f81854757b4f389d3bd22f595, user.id=80e142e6def84c2e8be21c9a73583e45, scope.system=all) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:13.845166 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ecc75-89f6-4a83-82e7-08215a7fb371 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:13.846829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ecc75-89f6-4a83-82e7-08215a7fb371 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:13.852384 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c656788-99e3-4011-97e4-403d1f56c992 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:13.854052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 133/281] 10.222.0.13 () {56 vars in 1308 bytes} [Sat Jun 25 15:53:13 2022] GET /identity/v3/role_assignments?user.id=80e142e6def84c2e8be21c9a73583e45&scope.system=all&role.id=8b9bae8f81854757b4f389d3bd22f595 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:13.855585 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb1faf-0405-4037-95c2-ab82f4aaa923 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:13.855872 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb1faf-0405-4037-95c2-ab82f4aaa923 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:13.856147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb1faf-0405-4037-95c2-ab82f4aaa923 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:13.859257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 134/282] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:13.864434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-5c656788-99e3-4011-97e4-403d1f56c992 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 25 15:53:13.864434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:13.864434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:13.864434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:13.864434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:13.864434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:13.864434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:13.864434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:13.864434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:13.864434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:13.864434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:13.864434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:13.864434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:13.864434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:13.864434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:13.864434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:13.865495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:13.865495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:13.865495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:13.865495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:13.865495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:13.865495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:13.865495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:13.865495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:13.865495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:13.865495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:13.865495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:13.865495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:13.865495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:13.865495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:13.865495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:13.865495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:13.866288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:13.866288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:13.866288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:13.866288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:13.866288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:13.866288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:13.866288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:13.866288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:13.866288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:13.866288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:13.866288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 25 15:53:13.866288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:13.866288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 149/283] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:13.870235 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb0806-9544-4941-8f3c-9e8f7864c1d6 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:13.870525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb0806-9544-4941-8f3c-9e8f7864c1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:13.870806 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb0806-9544-4941-8f3c-9e8f7864c1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:13.871418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c656788-99e3-4011-97e4-403d1f56c992 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:13.895305 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-cfbb0806-9544-4941-8f3c-9e8f7864c1d6 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:13.896908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-cfbb0806-9544-4941-8f3c-9e8f7864c1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:13.913920 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:13.954604 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:13.988814 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 135/284] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:14.008019 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c656788-99e3-4011-97e4-403d1f56c992 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:14.009105 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb0806-9544-4941-8f3c-9e8f7864c1d6 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:14.052982 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61baa119-5a39-4f04-94b0-13ac0dbbb21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:14.054113 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61baa119-5a39-4f04-94b0-13ac0dbbb21e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:14.054598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61baa119-5a39-4f04-94b0-13ac0dbbb21e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:14.055077 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61baa119-5a39-4f04-94b0-13ac0dbbb21e None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:14.056069 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61baa119-5a39-4f04-94b0-13ac0dbbb21e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:14.057276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61baa119-5a39-4f04-94b0-13ac0dbbb21e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:14.059908 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61baa119-5a39-4f04-94b0-13ac0dbbb21e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:14.071106 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 150/285] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:14.077643 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61baa119-5a39-4f04-94b0-13ac0dbbb21e None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:14.182150 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb0806-9544-4941-8f3c-9e8f7864c1d6 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:14.232523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61baa119-5a39-4f04-94b0-13ac0dbbb21e None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:14.265508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:14.269084 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a30e7-342d-4ab4-990b-f24b8418d685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:14.270693 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a30e7-342d-4ab4-990b-f24b8418d685 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:14.270974 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a30e7-342d-4ab4-990b-f24b8418d685 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:14.271250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a30e7-342d-4ab4-990b-f24b8418d685 None admin] PATH_INFO: `/v3/users/21b25f80c7d2495685cceff558ae23d8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:14.273863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a30e7-342d-4ab4-990b-f24b8418d685 None admin] RBAC: Authorizing `identity:get_user(user_id=21b25f80c7d2495685cceff558ae23d8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:14.276390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ae4ab-e0b1-43b6-beee-dc018eca3e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:14.277212 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ae4ab-e0b1-43b6-beee-dc018eca3e48 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:14.277575 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ae4ab-e0b1-43b6-beee-dc018eca3e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:14.277847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ae4ab-e0b1-43b6-beee-dc018eca3e48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:14.278556 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a30e7-342d-4ab4-990b-f24b8418d685 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:14.280230 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 151/286] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:14 2022] GET /identity/v3/users/21b25f80c7d2495685cceff558ae23d8 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:14.286196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a30e7-342d-4ab4-990b-f24b8418d685 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:14.302501 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ae4ab-e0b1-43b6-beee-dc018eca3e48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:14.311861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-5f8ae4ab-e0b1-43b6-beee-dc018eca3e48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:14.311861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:14.311861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:14.311861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:14.311861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:14.311861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:14.311861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:14.311861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:14.311861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:14.311861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:14.311861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:14.311861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:14.311861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:14.311861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:14.311861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:14.311861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:14.312956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:14.312956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:14.312956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:14.312956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:14.312956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:14.312956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:14.312956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:14.312956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:14.312956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:14.312956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:14.312956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:14.312956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:14.312956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:14.312956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:14.312956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:14.312956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:14.313969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:14.313969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:14.313969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:14.313969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:14.313969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:14.313969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:14.313969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:14.313969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:14.313969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:14.313969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:14.313969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:14.313969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:14.313969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 136/287] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:14.320402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ae4ab-e0b1-43b6-beee-dc018eca3e48 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:14.345052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:14.489594 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a30e7-342d-4ab4-990b-f24b8418d685 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:14.493754 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ae4ab-e0b1-43b6-beee-dc018eca3e48 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:14.534219 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91b86f-5f3a-45e5-81db-df5439caf28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:14.535209 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91b86f-5f3a-45e5-81db-df5439caf28a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:14.535721 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91b86f-5f3a-45e5-81db-df5439caf28a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:14.536092 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91b86f-5f3a-45e5-81db-df5439caf28a None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:14.537233 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91b86f-5f3a-45e5-81db-df5439caf28a None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:14.537637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91b86f-5f3a-45e5-81db-df5439caf28a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:14.539527 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91b86f-5f3a-45e5-81db-df5439caf28a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:14.549856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 137/288] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:14.557165 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91b86f-5f3a-45e5-81db-df5439caf28a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:14.558819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b2ae9-93ad-4ad9-b7b8-b7c96d0c7124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:14.559863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b2ae9-93ad-4ad9-b7b8-b7c96d0c7124 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:14.562667 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b2ae9-93ad-4ad9-b7b8-b7c96d0c7124 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:14.563124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b2ae9-93ad-4ad9-b7b8-b7c96d0c7124 None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:14.566158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b2ae9-93ad-4ad9-b7b8-b7c96d0c7124 None admin] RBAC: Authorizing `identity:get_project(project_id=3233ec59a0994511b0032830b4ded6c2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:14.567765 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b2ae9-93ad-4ad9-b7b8-b7c96d0c7124 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:14.574827 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 152/289] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:14 2022] GET /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2 => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:14.585297 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b2ae9-93ad-4ad9-b7b8-b7c96d0c7124 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:14.639616 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:14.682839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91b86f-5f3a-45e5-81db-df5439caf28a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:14.689663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:14.737109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924a703-ad5d-4868-aee2-65211945d86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:14.738008 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924a703-ad5d-4868-aee2-65211945d86c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:14.738284 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924a703-ad5d-4868-aee2-65211945d86c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:14.738555 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924a703-ad5d-4868-aee2-65211945d86c None admin] PATH_INFO: `/v3/users/5e42b1ec7f9f4871ab17c6f94ddd7f82` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:14.739953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924a703-ad5d-4868-aee2-65211945d86c None admin] RBAC: Authorizing `identity:get_user(user_id=5e42b1ec7f9f4871ab17c6f94ddd7f82)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:14.741379 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924a703-ad5d-4868-aee2-65211945d86c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:14.743387 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 138/290] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:14 2022] GET /identity/v3/users/5e42b1ec7f9f4871ab17c6f94ddd7f82 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:14.748600 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924a703-ad5d-4868-aee2-65211945d86c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:14.754663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b2ae9-93ad-4ad9-b7b8-b7c96d0c7124 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:14.816782 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ee846-93c0-49e1-8b39-1abac832dd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:14.817708 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ee846-93c0-49e1-8b39-1abac832dd84 None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:14.817985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ee846-93c0-49e1-8b39-1abac832dd84 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:14.818254 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ee846-93c0-49e1-8b39-1abac832dd84 None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2/users/21b25f80c7d2495685cceff558ae23d8/roles/f70856a334f7479fb00bac40f893e9f0` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:14.821989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ee846-93c0-49e1-8b39-1abac832dd84 None admin] RBAC: Authorizing `identity:create_grant(project_id=3233ec59a0994511b0032830b4ded6c2, user_id=21b25f80c7d2495685cceff558ae23d8, role_id=f70856a334f7479fb00bac40f893e9f0)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:14.823460 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ee846-93c0-49e1-8b39-1abac832dd84 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:14.850517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 153/291] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:53:14 2022] PUT /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2/users/21b25f80c7d2495685cceff558ae23d8/roles/f70856a334f7479fb00bac40f893e9f0 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:53:14.936759 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924a703-ad5d-4868-aee2-65211945d86c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:14.987987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13668c-f2b4-4582-9926-88b4806874e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:14.988715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13668c-f2b4-4582-9926-88b4806874e9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:14.988996 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13668c-f2b4-4582-9926-88b4806874e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:14.989551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13668c-f2b4-4582-9926-88b4806874e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:15.002137 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:15.079157 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13668c-f2b4-4582-9926-88b4806874e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:15.080734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13668c-f2b4-4582-9926-88b4806874e9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:15.189324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 139/292] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:15.194568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ee846-93c0-49e1-8b39-1abac832dd84 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:15.368510 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ee846-93c0-49e1-8b39-1abac832dd84 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:15.403675 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:15.426532 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca185b3-0a76-4e62-a0e1-c525d3553e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:15.427432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca185b3-0a76-4e62-a0e1-c525d3553e0e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:15.427716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca185b3-0a76-4e62-a0e1-c525d3553e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:15.427993 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca185b3-0a76-4e62-a0e1-c525d3553e0e None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:15.429268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca185b3-0a76-4e62-a0e1-c525d3553e0e None admin] RBAC: Authorizing `identity:get_project(project_id=3233ec59a0994511b0032830b4ded6c2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:15.430866 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca185b3-0a76-4e62-a0e1-c525d3553e0e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:15.433353 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 154/293] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:15 2022] GET /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:15.439429 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13668c-f2b4-4582-9926-88b4806874e9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:15.507575 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:15.586651 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13668c-f2b4-4582-9926-88b4806874e9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:15.623895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98971597-fad9-4075-9c50-24d5437c21da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:15.624932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98971597-fad9-4075-9c50-24d5437c21da None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:15.625402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98971597-fad9-4075-9c50-24d5437c21da None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:15.625859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98971597-fad9-4075-9c50-24d5437c21da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:15.626897 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98971597-fad9-4075-9c50-24d5437c21da None admin] RBAC: Adding query filter params (role.id=8b9bae8f81854757b4f389d3bd22f595, scope.project.id=3233ec59a0994511b0032830b4ded6c2, user.id=5e42b1ec7f9f4871ab17c6f94ddd7f82) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:15.627414 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98971597-fad9-4075-9c50-24d5437c21da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:15.629085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98971597-fad9-4075-9c50-24d5437c21da None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:15.635839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 140/294] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:53:15 2022] GET /identity/v3/role_assignments?user.id=5e42b1ec7f9f4871ab17c6f94ddd7f82&scope.project.id=3233ec59a0994511b0032830b4ded6c2&role.id=8b9bae8f81854757b4f389d3bd22f595 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:16.058602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5dafb-6876-4010-a4ae-3b73f04e6d8a None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:16.059317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5dafb-6876-4010-a4ae-3b73f04e6d8a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:16.059792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5dafb-6876-4010-a4ae-3b73f04e6d8a None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:16.061668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 155/295] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:16.067990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed74e5-7ca3-443e-b38a-2aca9f6cbc7a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:16.068902 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed74e5-7ca3-443e-b38a-2aca9f6cbc7a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:16.069382 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed74e5-7ca3-443e-b38a-2aca9f6cbc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:16.071583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845c030-ee94-43ef-a900-4dceedfc0a21 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:16.071924 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845c030-ee94-43ef-a900-4dceedfc0a21 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:16.072207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845c030-ee94-43ef-a900-4dceedfc0a21 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:16.073581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 156/296] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:16.079209 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c894d89-1933-4d22-bb46-4029fcf55ef1 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:16.079496 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c894d89-1933-4d22-bb46-4029fcf55ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:16.079776 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c894d89-1933-4d22-bb46-4029fcf55ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:16.090027 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-77ed74e5-7ca3-443e-b38a-2aca9f6cbc7a None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:16.091492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-77ed74e5-7ca3-443e-b38a-2aca9f6cbc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:16.097815 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9c894d89-1933-4d22-bb46-4029fcf55ef1 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:16.098984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9c894d89-1933-4d22-bb46-4029fcf55ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:16.177880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 157/297] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:53:16.188502 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 141/298] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:16.200913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c894d89-1933-4d22-bb46-4029fcf55ef1 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:16.208856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed74e5-7ca3-443e-b38a-2aca9f6cbc7a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:16.254459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:16.317402 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c894d89-1933-4d22-bb46-4029fcf55ef1 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:16.338823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:16.342216 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed74e5-7ca3-443e-b38a-2aca9f6cbc7a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:16.362889 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfc863-3292-4485-8f61-4c8d155a464b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:16.364283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfc863-3292-4485-8f61-4c8d155a464b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:16.365011 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfc863-3292-4485-8f61-4c8d155a464b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:16.365827 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfc863-3292-4485-8f61-4c8d155a464b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:16.367212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bfc863-3292-4485-8f61-4c8d155a464b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:16.369451 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bfc863-3292-4485-8f61-4c8d155a464b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:16.372708 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 158/299] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:16.375932 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01414b3b-ecd8-41b8-b296-f3463af7fe4f None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:16.376512 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01414b3b-ecd8-41b8-b296-f3463af7fe4f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:16.376993 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01414b3b-ecd8-41b8-b296-f3463af7fe4f None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:16.379189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 159/300] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:16.380818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01414b3b-ecd8-41b8-b296-f3463af7fe4f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:16.387421 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b008093-6364-4e14-a5c0-47c4a2efb1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:16.388222 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b008093-6364-4e14-a5c0-47c4a2efb1ab None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:16.388549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b008093-6364-4e14-a5c0-47c4a2efb1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:16.388824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b008093-6364-4e14-a5c0-47c4a2efb1ab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:16.399480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b008093-6364-4e14-a5c0-47c4a2efb1ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:16.407345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-5b008093-6364-4e14-a5c0-47c4a2efb1ab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 25 15:53:16.407345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:16.407345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:16.407345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:16.407345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:16.407345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:16.407345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:16.407345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:16.407345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:16.407345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:16.407345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:16.407345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:16.407345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:16.407345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:16.407345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:16.407345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:16.408210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:16.408210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:16.408210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:16.408210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:16.408210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:16.408210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:16.408210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:16.408210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:16.408210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:16.408210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:16.408210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:16.408210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:16.408210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:16.408210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:16.408210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:16.408210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:16.413885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:16.413885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:16.413885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:16.413885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:16.413885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:16.413885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:16.413885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:16.413885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:16.413885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:16.413885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:16.413885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 25 15:53:16.413885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:16.413885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 142/301] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:16.413885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab75fb-b093-4121-a354-310626597367 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:16.414582 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab75fb-b093-4121-a354-310626597367 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:16.414582 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab75fb-b093-4121-a354-310626597367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:16.427959 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-9dab75fb-b093-4121-a354-310626597367 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:16.429227 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9dab75fb-b093-4121-a354-310626597367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:16.502000 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01414b3b-ecd8-41b8-b296-f3463af7fe4f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:16.510268 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 143/302] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:16.512805 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab75fb-b093-4121-a354-310626597367 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:16.537970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b28731-d96f-4702-824f-1e0ebc152402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:16.538641 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b28731-d96f-4702-824f-1e0ebc152402 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:16.538913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b28731-d96f-4702-824f-1e0ebc152402 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:16.539186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b28731-d96f-4702-824f-1e0ebc152402 None admin] PATH_INFO: `/v3/groups` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:16.539869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b28731-d96f-4702-824f-1e0ebc152402 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:16.541448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b28731-d96f-4702-824f-1e0ebc152402 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:16.541764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:16.558005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 160/303] 10.222.0.13 () {60 vars in 1134 bytes} [Sat Jun 25 15:53:16 2022] POST /identity/v3/groups => generated 229 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:53:16.569696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b28731-d96f-4702-824f-1e0ebc152402 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:16.634964 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:16.642955 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dab75fb-b093-4121-a354-310626597367 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:16.694586 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d7e80-0e18-4592-bf1b-3db0cedc8606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:16.695325 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d7e80-0e18-4592-bf1b-3db0cedc8606 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:16.695720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d7e80-0e18-4592-bf1b-3db0cedc8606 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:16.696057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d7e80-0e18-4592-bf1b-3db0cedc8606 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:16.696946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d7e80-0e18-4592-bf1b-3db0cedc8606 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:16.697320 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d7e80-0e18-4592-bf1b-3db0cedc8606 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:16.699480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d7e80-0e18-4592-bf1b-3db0cedc8606 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:16.708740 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 144/304] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:16.711565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b28731-d96f-4702-824f-1e0ebc152402 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:16.715389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d7e80-0e18-4592-bf1b-3db0cedc8606 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:16.753733 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef1840-9fb0-419f-b39b-888583ba0604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:16.754778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef1840-9fb0-419f-b39b-888583ba0604 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:16.755196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef1840-9fb0-419f-b39b-888583ba0604 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:16.755628 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef1840-9fb0-419f-b39b-888583ba0604 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:16.769138 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef1840-9fb0-419f-b39b-888583ba0604 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:16.777722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-38ef1840-9fb0-419f-b39b-888583ba0604 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:16.777722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:16.777722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:16.777722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:16.777722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:16.777722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:16.777722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:16.777722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:16.777722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:16.777722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:16.777722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:16.777722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:16.777722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:16.777722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:16.777722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:16.777722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:16.778617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:16.778617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:16.778617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:16.778617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:16.778617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:16.778617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:16.778617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:16.778617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:16.778617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:16.778617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:16.778617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:16.778617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:16.778617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:16.778617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:16.778617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:16.778617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:16.779383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:16.779383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:16.779383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:16.779383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:16.779383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:16.779383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:16.779383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:16.779383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:16.779383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:16.779383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:16.779383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:16.779383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:16.779383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 161/305] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:16.787266 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef1840-9fb0-419f-b39b-888583ba0604 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:16.813514 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:16.830019 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d7e80-0e18-4592-bf1b-3db0cedc8606 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:16.863964 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca71b3-2ae6-4a05-8b95-e1beed83124e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:16.864961 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca71b3-2ae6-4a05-8b95-e1beed83124e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:16.865573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca71b3-2ae6-4a05-8b95-e1beed83124e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:16.865573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca71b3-2ae6-4a05-8b95-e1beed83124e None admin] PATH_INFO: `/v3/users/426b2dff36ce4570ba8eb544c4d88fd2` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:16.880669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca71b3-2ae6-4a05-8b95-e1beed83124e None admin] RBAC: Authorizing `identity:get_user(user_id=426b2dff36ce4570ba8eb544c4d88fd2)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:16.882211 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca71b3-2ae6-4a05-8b95-e1beed83124e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:16.883938 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 145/306] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:16 2022] GET /identity/v3/users/426b2dff36ce4570ba8eb544c4d88fd2 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:16.889873 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca71b3-2ae6-4a05-8b95-e1beed83124e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:16.898204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef1840-9fb0-419f-b39b-888583ba0604 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:16.906871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:16.943300 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db36c6a-9e47-44a5-9958-f3b9b1ee264d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:16.944210 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db36c6a-9e47-44a5-9958-f3b9b1ee264d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:16.944684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db36c6a-9e47-44a5-9958-f3b9b1ee264d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:16.945092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db36c6a-9e47-44a5-9958-f3b9b1ee264d None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:16.946121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db36c6a-9e47-44a5-9958-f3b9b1ee264d None admin] RBAC: Adding query filter params (name=member) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:16.946698 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db36c6a-9e47-44a5-9958-f3b9b1ee264d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:16.948326 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db36c6a-9e47-44a5-9958-f3b9b1ee264d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:16.958067 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 162/307] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:16.963653 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db36c6a-9e47-44a5-9958-f3b9b1ee264d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:17.008346 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca71b3-2ae6-4a05-8b95-e1beed83124e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:17.045695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029604f-7591-4db4-b173-c59762d04db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:17.046362 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029604f-7591-4db4-b173-c59762d04db9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:17.046642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029604f-7591-4db4-b173-c59762d04db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:17.046914 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029604f-7591-4db4-b173-c59762d04db9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:17.047751 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029604f-7591-4db4-b173-c59762d04db9 None admin] RBAC: Adding query filter params (role.id=f70856a334f7479fb00bac40f893e9f0, user.id=426b2dff36ce4570ba8eb544c4d88fd2, scope.system=all) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:17.048057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029604f-7591-4db4-b173-c59762d04db9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:17.049630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029604f-7591-4db4-b173-c59762d04db9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:17.055274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 146/308] 10.222.0.13 () {56 vars in 1308 bytes} [Sat Jun 25 15:53:16 2022] GET /identity/v3/role_assignments?user.id=426b2dff36ce4570ba8eb544c4d88fd2&scope.system=all&role.id=f70856a334f7479fb00bac40f893e9f0 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:17.074522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db36c6a-9e47-44a5-9958-f3b9b1ee264d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:17.115136 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ebe98-191a-4d1f-b2d1-7c789dec06e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:17.116712 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ebe98-191a-4d1f-b2d1-7c789dec06e0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:17.117171 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ebe98-191a-4d1f-b2d1-7c789dec06e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:17.117634 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ebe98-191a-4d1f-b2d1-7c789dec06e0 None admin] PATH_INFO: `/v3/users/80e142e6def84c2e8be21c9a73583e45` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:17.123399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ebe98-191a-4d1f-b2d1-7c789dec06e0 None admin] RBAC: Authorizing `identity:get_user(user_id=80e142e6def84c2e8be21c9a73583e45)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:17.125302 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ebe98-191a-4d1f-b2d1-7c789dec06e0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:17.127360 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 163/309] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:16 2022] GET /identity/v3/users/80e142e6def84c2e8be21c9a73583e45 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:17.133300 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029604f-7591-4db4-b173-c59762d04db9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:17.146587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:17.244193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029604f-7591-4db4-b173-c59762d04db9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:17.280944 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01df71-172d-4607-8cc9-b76a3100dfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:17.282194 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01df71-172d-4607-8cc9-b76a3100dfbd None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:17.282665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01df71-172d-4607-8cc9-b76a3100dfbd None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:17.283154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01df71-172d-4607-8cc9-b76a3100dfbd None admin] PATH_INFO: `/v3/system/users/80e142e6def84c2e8be21c9a73583e45/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:17.284876 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01df71-172d-4607-8cc9-b76a3100dfbd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=80e142e6def84c2e8be21c9a73583e45, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:17.286442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01df71-172d-4607-8cc9-b76a3100dfbd None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:17.297681 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 147/310] 10.222.0.13 () {58 vars in 1334 bytes} [Sat Jun 25 15:53:17 2022] PUT /identity/v3/system/users/80e142e6def84c2e8be21c9a73583e45/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:53:17.381518 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126efd19-9e11-4d81-92dc-6d3db8c4d7ef None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:17.382501 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126efd19-9e11-4d81-92dc-6d3db8c4d7ef None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:17.382976 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126efd19-9e11-4d81-92dc-6d3db8c4d7ef None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:17.385086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 164/311] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:17.391083 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5efc7-b01b-4e48-9bc0-830f9c1e3479 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:17.391674 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5efc7-b01b-4e48-9bc0-830f9c1e3479 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:17.391967 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5efc7-b01b-4e48-9bc0-830f9c1e3479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:17.410142 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-9ca5efc7-b01b-4e48-9bc0-830f9c1e3479 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:17.411661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9ca5efc7-b01b-4e48-9bc0-830f9c1e3479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:17.484054 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 148/312] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:17.507968 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126efd19-9e11-4d81-92dc-6d3db8c4d7ef None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:17.549422 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:17.621071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126efd19-9e11-4d81-92dc-6d3db8c4d7ef None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:17.657119 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9523fcd3-2836-414a-989c-06fe9361fa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:17.657837 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523fcd3-2836-414a-989c-06fe9361fa65 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:17.658118 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523fcd3-2836-414a-989c-06fe9361fa65 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:17.658389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523fcd3-2836-414a-989c-06fe9361fa65 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:17.669147 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523fcd3-2836-414a-989c-06fe9361fa65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:17.676727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-9523fcd3-2836-414a-989c-06fe9361fa65 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 25 15:53:17.676727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:17.676727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:17.676727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:17.676727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:17.676727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:17.676727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:17.676727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:17.676727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:17.676727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:17.676727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:17.676727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:17.676727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:17.676727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:17.676727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:17.676727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:17.677609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:17.677609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:17.677609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:17.677609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:17.677609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:17.677609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:17.677609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:17.677609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:17.677609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:17.677609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:17.677609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:17.677609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:17.677609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:17.677609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:17.677609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:17.677609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:17.678399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:17.678399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:17.678399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:17.678399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:17.678399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:17.678399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:17.678399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:17.678399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:17.678399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:17.678399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:17.678399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 25 15:53:17.678399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:17.678399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 165/313] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:17.685448 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5efc7-b01b-4e48-9bc0-830f9c1e3479 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:17.687900 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:17.789048 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5efc7-b01b-4e48-9bc0-830f9c1e3479 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:17.829585 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929518f-a4b5-4752-8868-65606fafd73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:17.830323 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929518f-a4b5-4752-8868-65606fafd73a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:17.830604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929518f-a4b5-4752-8868-65606fafd73a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:17.830928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929518f-a4b5-4752-8868-65606fafd73a None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:17.831691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929518f-a4b5-4752-8868-65606fafd73a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:17.831993 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929518f-a4b5-4752-8868-65606fafd73a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:17.833408 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929518f-a4b5-4752-8868-65606fafd73a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:17.843110 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 149/314] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:17.849365 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9523fcd3-2836-414a-989c-06fe9361fa65 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:17.954505 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:17.960871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9523fcd3-2836-414a-989c-06fe9361fa65 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:17.994732 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e214abc-9a6e-4670-b1ec-c9c512278b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:17.995618 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e214abc-9a6e-4670-b1ec-c9c512278b18 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:17.995897 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e214abc-9a6e-4670-b1ec-c9c512278b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:17.996171 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e214abc-9a6e-4670-b1ec-c9c512278b18 None admin] PATH_INFO: `/v3/users/21b25f80c7d2495685cceff558ae23d8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:17.997400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e214abc-9a6e-4670-b1ec-c9c512278b18 None admin] RBAC: Authorizing `identity:get_user(user_id=21b25f80c7d2495685cceff558ae23d8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:17.998875 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e214abc-9a6e-4670-b1ec-c9c512278b18 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:18.000556 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 166/315] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:17 2022] GET /identity/v3/users/21b25f80c7d2495685cceff558ae23d8 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:18.006392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929518f-a4b5-4752-8868-65606fafd73a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:18.071074 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:18.112001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929518f-a4b5-4752-8868-65606fafd73a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:18.146580 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1968bdd-f57b-4762-807a-b6b6d5ec59e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:18.147936 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1968bdd-f57b-4762-807a-b6b6d5ec59e6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:18.148384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1968bdd-f57b-4762-807a-b6b6d5ec59e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:18.148814 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1968bdd-f57b-4762-807a-b6b6d5ec59e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:18.218707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1968bdd-f57b-4762-807a-b6b6d5ec59e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:18.220401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1968bdd-f57b-4762-807a-b6b6d5ec59e6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:18.286323 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:18.322551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 150/316] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:18.329562 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e214abc-9a6e-4670-b1ec-c9c512278b18 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:18.437801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e214abc-9a6e-4670-b1ec-c9c512278b18 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:18.472662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675335f-7fcb-481f-93af-58fbda54edd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:18.473458 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675335f-7fcb-481f-93af-58fbda54edd2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:18.473734 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675335f-7fcb-481f-93af-58fbda54edd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:18.474009 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675335f-7fcb-481f-93af-58fbda54edd2 None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:18.475256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675335f-7fcb-481f-93af-58fbda54edd2 None admin] RBAC: Authorizing `identity:get_project(project_id=3233ec59a0994511b0032830b4ded6c2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:18.476756 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675335f-7fcb-481f-93af-58fbda54edd2 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:18.478967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 167/317] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:18 2022] GET /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:18.485604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1968bdd-f57b-4762-807a-b6b6d5ec59e6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:18.600431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1968bdd-f57b-4762-807a-b6b6d5ec59e6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:18.635050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c34fd-7236-4a46-89a3-4a077ae5f963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:18.635712 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c34fd-7236-4a46-89a3-4a077ae5f963 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:18.635990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c34fd-7236-4a46-89a3-4a077ae5f963 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:18.636265 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c34fd-7236-4a46-89a3-4a077ae5f963 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:18.637131 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c34fd-7236-4a46-89a3-4a077ae5f963 None admin] RBAC: Adding query filter params (role.id=f70856a334f7479fb00bac40f893e9f0, scope.project.id=3233ec59a0994511b0032830b4ded6c2, user.id=21b25f80c7d2495685cceff558ae23d8) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:18.637441 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c34fd-7236-4a46-89a3-4a077ae5f963 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:18.638876 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c34fd-7236-4a46-89a3-4a077ae5f963 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:18.645262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 151/318] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:53:18 2022] GET /identity/v3/role_assignments?user.id=21b25f80c7d2495685cceff558ae23d8&scope.project.id=3233ec59a0994511b0032830b4ded6c2&role.id=f70856a334f7479fb00bac40f893e9f0 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:18.771541 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7dd3a7-7a66-4603-bde0-37e49557ed0c None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:18.772161 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7dd3a7-7a66-4603-bde0-37e49557ed0c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:18.772768 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7dd3a7-7a66-4603-bde0-37e49557ed0c None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:18.774668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 168/319] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:18.781648 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ccfe9-27a7-41f1-bfc7-64e9f38ebe2a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:18.782278 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ccfe9-27a7-41f1-bfc7-64e9f38ebe2a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:18.782770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ccfe9-27a7-41f1-bfc7-64e9f38ebe2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:18.802060 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-6c9ccfe9-27a7-41f1-bfc7-64e9f38ebe2a None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:18.803589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6c9ccfe9-27a7-41f1-bfc7-64e9f38ebe2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:18.830264 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:18.881699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 152/320] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:18.906590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7dd3a7-7a66-4603-bde0-37e49557ed0c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:18.941182 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:19.017675 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7dd3a7-7a66-4603-bde0-37e49557ed0c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:19.053041 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f2f53-8852-4d42-adf8-f3457d5e4f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:19.053759 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f2f53-8852-4d42-adf8-f3457d5e4f0e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:19.054041 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f2f53-8852-4d42-adf8-f3457d5e4f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:19.054309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f2f53-8852-4d42-adf8-f3457d5e4f0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:19.055518 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78f2f53-8852-4d42-adf8-f3457d5e4f0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:19.057605 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78f2f53-8852-4d42-adf8-f3457d5e4f0e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:19.063784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 169/321] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:19.065162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ccfe9-27a7-41f1-bfc7-64e9f38ebe2a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:19.168060 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ccfe9-27a7-41f1-bfc7-64e9f38ebe2a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:19.202376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e235d-b7f5-4987-9376-7cfb77cb4fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:19.203052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e235d-b7f5-4987-9376-7cfb77cb4fe1 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:19.203334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e235d-b7f5-4987-9376-7cfb77cb4fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:19.203606 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e235d-b7f5-4987-9376-7cfb77cb4fe1 None admin] PATH_INFO: `/v3/groups` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:19.204301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e235d-b7f5-4987-9376-7cfb77cb4fe1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:19.205776 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e235d-b7f5-4987-9376-7cfb77cb4fe1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:19.221498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 153/322] 10.222.0.13 () {60 vars in 1134 bytes} [Sat Jun 25 15:53:19 2022] POST /identity/v3/groups => generated 226 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:53:19.277434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8a50e-b729-40b4-b84e-4e6dcd657cd9 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:19.277689 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8a50e-b729-40b4-b84e-4e6dcd657cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:19.277983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8a50e-b729-40b4-b84e-4e6dcd657cd9 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:19.279360 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 170/323] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:19.285546 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb3569-735a-43bc-be07-4f35e6c494a4 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:19.285832 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb3569-735a-43bc-be07-4f35e6c494a4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:19.286094 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb3569-735a-43bc-be07-4f35e6c494a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:19.304829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-47cb3569-735a-43bc-be07-4f35e6c494a4 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:19.305961 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-47cb3569-735a-43bc-be07-4f35e6c494a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:19.321834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:19.385197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 154/324] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:19.407367 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8a50e-b729-40b4-b84e-4e6dcd657cd9 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:19.429212 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f705fa-98ea-4372-bc63-1141c0bb4d0a None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:19.429750 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f705fa-98ea-4372-bc63-1141c0bb4d0a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:19.430021 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f705fa-98ea-4372-bc63-1141c0bb4d0a None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:19.431448 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 155/325] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:53:19.436801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7eade-062f-4ecd-9824-9e092dac2aa3 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:19.437085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7eade-062f-4ecd-9824-9e092dac2aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:19.437363 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7eade-062f-4ecd-9824-9e092dac2aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:19.454342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-bbe7eade-062f-4ecd-9824-9e092dac2aa3 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:19.455523 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bbe7eade-062f-4ecd-9824-9e092dac2aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:19.518265 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:19.522070 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8a50e-b729-40b4-b84e-4e6dcd657cd9 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:19.547362 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 156/326] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:53:19.569639 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645ca22-bcd7-499f-89f0-3634ef6cea69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:19.570473 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645ca22-bcd7-499f-89f0-3634ef6cea69 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:19.570768 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645ca22-bcd7-499f-89f0-3634ef6cea69 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:19.571044 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645ca22-bcd7-499f-89f0-3634ef6cea69 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:19.573587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7eade-062f-4ecd-9824-9e092dac2aa3 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:19.584379 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a645ca22-bcd7-499f-89f0-3634ef6cea69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:19.594097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-a645ca22-bcd7-499f-89f0-3634ef6cea69 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 25 15:53:19.594097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:19.594097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:19.594097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:19.594097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:19.594097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:19.594097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:19.594097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:19.594097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:19.594097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:19.594097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:19.594097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:19.594097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:19.594097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:19.594097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:19.594097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:19.595174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:19.595174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:19.595174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:19.595174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:19.595174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:19.595174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:19.595174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:19.595174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:19.595174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:19.595174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:19.595174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:19.595174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:19.595174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:19.595174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:19.595174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:19.595174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:19.596213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:19.596213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:19.596213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:19.596213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:19.596213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:19.596213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:19.596213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:19.596213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:19.596213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:19.596213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:19.596213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 25 15:53:19.596213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:19.596213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 171/327] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:19.602193 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645ca22-bcd7-499f-89f0-3634ef6cea69 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:19.672237 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:19.703111 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7eade-062f-4ecd-9824-9e092dac2aa3 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:19.740637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645ca22-bcd7-499f-89f0-3634ef6cea69 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:19.749349 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344876c-ba49-4b54-8eb7-6d5a367c3ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:19.750155 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344876c-ba49-4b54-8eb7-6d5a367c3ade None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:19.750780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344876c-ba49-4b54-8eb7-6d5a367c3ade None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:19.751059 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344876c-ba49-4b54-8eb7-6d5a367c3ade None admin] PATH_INFO: `/v3/roles/member` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:19.763793 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344876c-ba49-4b54-8eb7-6d5a367c3ade None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:19.771908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-e344876c-ba49-4b54-8eb7-6d5a367c3ade None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:19.771908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:19.771908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:19.771908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:19.771908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:19.771908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:19.771908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:19.771908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:19.771908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:19.771908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:19.771908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:19.771908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:19.771908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:19.771908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:19.771908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:19.771908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:19.772841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:19.772841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:19.772841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:19.772841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:19.772841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:19.772841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:19.772841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:19.772841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:19.772841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:19.772841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:19.772841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:19.772841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:19.772841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:19.772841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:19.772841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:19.772841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:19.773630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:19.773630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:19.773630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:19.773630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:19.773630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:19.773630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:19.773630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:19.773630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:19.773630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:19.773630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:19.773630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:19.773630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:19.776519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 157/328] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:19.779527 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344876c-ba49-4b54-8eb7-6d5a367c3ade None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:19.780522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4b1d0-7d09-41b2-88cc-ee6a79d78291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:19.781181 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4b1d0-7d09-41b2-88cc-ee6a79d78291 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:19.781462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4b1d0-7d09-41b2-88cc-ee6a79d78291 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:19.781735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4b1d0-7d09-41b2-88cc-ee6a79d78291 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:19.782488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4b1d0-7d09-41b2-88cc-ee6a79d78291 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:19.782791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4b1d0-7d09-41b2-88cc-ee6a79d78291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:19.784220 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4b1d0-7d09-41b2-88cc-ee6a79d78291 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:19.796680 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 172/329] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:19.799586 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4b1d0-7d09-41b2-88cc-ee6a79d78291 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:19.826238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:19.904508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344876c-ba49-4b54-8eb7-6d5a367c3ade None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:19.911469 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4b1d0-7d09-41b2-88cc-ee6a79d78291 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:19.944318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf90de4-0ff0-4deb-a44f-2c71bf800509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:19.945024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf90de4-0ff0-4deb-a44f-2c71bf800509 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:19.945376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf90de4-0ff0-4deb-a44f-2c71bf800509 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:19.945992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf90de4-0ff0-4deb-a44f-2c71bf800509 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:19.947264 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf90de4-0ff0-4deb-a44f-2c71bf800509 None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:19.947582 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf90de4-0ff0-4deb-a44f-2c71bf800509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:19.949080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf90de4-0ff0-4deb-a44f-2c71bf800509 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:19.950923 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35dec7c-83de-4e8a-ac6b-6def808851b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:19.951995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35dec7c-83de-4e8a-ac6b-6def808851b9 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:19.952524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35dec7c-83de-4e8a-ac6b-6def808851b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:19.952969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35dec7c-83de-4e8a-ac6b-6def808851b9 None admin] PATH_INFO: `/v3/users/426b2dff36ce4570ba8eb544c4d88fd2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:19.955077 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35dec7c-83de-4e8a-ac6b-6def808851b9 None admin] RBAC: Authorizing `identity:get_user(user_id=426b2dff36ce4570ba8eb544c4d88fd2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:19.956738 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35dec7c-83de-4e8a-ac6b-6def808851b9 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:19.958139 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 158/330] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:19.958487 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:19.960444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 173/331] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:19 2022] GET /identity/v3/users/426b2dff36ce4570ba8eb544c4d88fd2 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:19.964388 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf90de4-0ff0-4deb-a44f-2c71bf800509 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:19.965127 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35dec7c-83de-4e8a-ac6b-6def808851b9 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:20.076092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:20.087956 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35dec7c-83de-4e8a-ac6b-6def808851b9 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:20.103394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf90de4-0ff0-4deb-a44f-2c71bf800509 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:20.145212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e3352-5f0c-4ab1-b983-3596ce19b0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:20.146511 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755243c-6506-464a-9e33-c4e2e138ef86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:20.146852 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e3352-5f0c-4ab1-b983-3596ce19b0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:20.147330 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e3352-5f0c-4ab1-b983-3596ce19b0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:20.147586 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755243c-6506-464a-9e33-c4e2e138ef86 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:20.147913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e3352-5f0c-4ab1-b983-3596ce19b0c7 None admin] PATH_INFO: `/v3/users/80e142e6def84c2e8be21c9a73583e45` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:20.148016 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755243c-6506-464a-9e33-c4e2e138ef86 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:20.148306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755243c-6506-464a-9e33-c4e2e138ef86 None admin] PATH_INFO: `/v3/system/users/426b2dff36ce4570ba8eb544c4d88fd2/roles/f70856a334f7479fb00bac40f893e9f0` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:20.150172 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755243c-6506-464a-9e33-c4e2e138ef86 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=426b2dff36ce4570ba8eb544c4d88fd2, role_id=f70856a334f7479fb00bac40f893e9f0)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:20.150657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e3352-5f0c-4ab1-b983-3596ce19b0c7 None admin] RBAC: Authorizing `identity:get_user(user_id=80e142e6def84c2e8be21c9a73583e45)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:20.151480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755243c-6506-464a-9e33-c4e2e138ef86 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:20.152387 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e3352-5f0c-4ab1-b983-3596ce19b0c7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:20.154511 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 174/332] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:19 2022] GET /identity/v3/users/80e142e6def84c2e8be21c9a73583e45 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:20.160262 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e3352-5f0c-4ab1-b983-3596ce19b0c7 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:20.163081 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 159/333] 10.222.0.13 () {58 vars in 1334 bytes} [Sat Jun 25 15:53:19 2022] PUT /identity/v3/system/users/426b2dff36ce4570ba8eb544c4d88fd2/roles/f70856a334f7479fb00bac40f893e9f0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:53:20.287781 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e3352-5f0c-4ab1-b983-3596ce19b0c7 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:20.323509 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf440914-be3f-40e6-a5ff-79b0eeac5064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:20.324199 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf440914-be3f-40e6-a5ff-79b0eeac5064 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:20.324512 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf440914-be3f-40e6-a5ff-79b0eeac5064 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:20.324792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf440914-be3f-40e6-a5ff-79b0eeac5064 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:20.325656 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf440914-be3f-40e6-a5ff-79b0eeac5064 None admin] RBAC: Adding query filter params (role.id=8b9bae8f81854757b4f389d3bd22f595, user.id=80e142e6def84c2e8be21c9a73583e45, scope.system=all) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:20.325969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf440914-be3f-40e6-a5ff-79b0eeac5064 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:20.327471 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf440914-be3f-40e6-a5ff-79b0eeac5064 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:20.341416 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 175/334] 10.222.0.13 () {56 vars in 1308 bytes} [Sat Jun 25 15:53:20 2022] GET /identity/v3/role_assignments?user.id=80e142e6def84c2e8be21c9a73583e45&scope.system=all&role.id=8b9bae8f81854757b4f389d3bd22f595 => generated 510 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:21.960683 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c51e20-fc19-4f7c-b584-675ddea2b18a None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:21.961652 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c51e20-fc19-4f7c-b584-675ddea2b18a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:21.962121 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c51e20-fc19-4f7c-b584-675ddea2b18a None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:21.963816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 160/335] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:21.970190 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb8898-7b7a-4bb3-8595-aa46947923ca None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:21.970638 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb8898-7b7a-4bb3-8595-aa46947923ca None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:21.970818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb8898-7b7a-4bb3-8595-aa46947923ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:21.993442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-48fb8898-7b7a-4bb3-8595-aa46947923ca None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:21.995355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-48fb8898-7b7a-4bb3-8595-aa46947923ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:22.007444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:22.079044 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 176/336] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:22.107040 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c51e20-fc19-4f7c-b584-675ddea2b18a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:22.138088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6238ac-5ad5-422a-bb08-a1b98a5a3906 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:22.138788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6238ac-5ad5-422a-bb08-a1b98a5a3906 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:22.139284 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6238ac-5ad5-422a-bb08-a1b98a5a3906 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:22.141189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 177/337] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:53:22.152986 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c728d54-d6ba-4f06-8bdd-fbb3ac8ff91e None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:22.153686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c728d54-d6ba-4f06-8bdd-fbb3ac8ff91e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:22.154154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c728d54-d6ba-4f06-8bdd-fbb3ac8ff91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:22.184336 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-2c728d54-d6ba-4f06-8bdd-fbb3ac8ff91e None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:22.185897 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2c728d54-d6ba-4f06-8bdd-fbb3ac8ff91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:22.206680 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:22.288445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 178/338] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 25 15:53:22.302262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c51e20-fc19-4f7c-b584-675ddea2b18a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:22.328737 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c728d54-d6ba-4f06-8bdd-fbb3ac8ff91e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:22.357819 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aaa70c-9860-47d4-8107-e206ee311194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:22.358653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaa70c-9860-47d4-8107-e206ee311194 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:22.358933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaa70c-9860-47d4-8107-e206ee311194 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:22.359206 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aaa70c-9860-47d4-8107-e206ee311194 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:22.378482 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aaa70c-9860-47d4-8107-e206ee311194 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:22.390483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-b9aaa70c-9860-47d4-8107-e206ee311194 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:22.390483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:22.390483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:22.390483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:22.390483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:22.390483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:22.390483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:22.390483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:22.390483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:22.390483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:22.390483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:22.390483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:22.390483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:22.390483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:22.390483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:22.390483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:22.391698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:22.391698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:22.391698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:22.391698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:22.391698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:22.391698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:22.391698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:22.391698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:22.391698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:22.391698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:22.391698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:22.391698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:22.391698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:22.391698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:22.391698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:22.391698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:22.392535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:22.392535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:22.392535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:22.392535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:22.392535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:22.392535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:22.392535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:22.392535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:22.392535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:22.392535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:22.392535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:22.392535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:22.392535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 161/339] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:53:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:22.405424 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aaa70c-9860-47d4-8107-e206ee311194 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:22.463890 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c728d54-d6ba-4f06-8bdd-fbb3ac8ff91e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:22.483295 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:22.515729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976adc6c-0db7-45ad-b9f5-23b0f76e126e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:22.517344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976adc6c-0db7-45ad-b9f5-23b0f76e126e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:22.517821 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976adc6c-0db7-45ad-b9f5-23b0f76e126e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:22.518284 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976adc6c-0db7-45ad-b9f5-23b0f76e126e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:22.549012 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aaa70c-9860-47d4-8107-e206ee311194 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:22.549211 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976adc6c-0db7-45ad-b9f5-23b0f76e126e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:22.556970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-976adc6c-0db7-45ad-b9f5-23b0f76e126e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:22.556970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:22.556970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:22.556970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:22.556970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:22.556970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:22.556970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:22.556970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:22.556970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:22.556970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:22.556970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:22.556970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:22.556970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:22.556970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:22.556970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:22.556970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:22.557874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:22.557874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:22.557874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:22.557874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:22.557874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:22.557874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:22.557874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:22.557874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:22.557874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:22.557874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:22.557874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:22.557874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:22.557874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:22.557874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:22.557874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:22.557874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:22.558646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:22.558646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:22.558646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:22.558646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:22.558646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:22.558646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:22.558646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:22.558646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:22.558646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:22.558646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:22.558646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:22.558646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:22.564661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 179/340] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:22.566059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f1332-9967-4d1d-8914-07a844034ca4 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:22.566342 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f1332-9967-4d1d-8914-07a844034ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:22.566618 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f1332-9967-4d1d-8914-07a844034ca4 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:22.567974 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 180/341] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:22.576544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c89cd7-d926-4d93-a7fd-8d8f0306c65e None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:22.576835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c89cd7-d926-4d93-a7fd-8d8f0306c65e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:22.577101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c89cd7-d926-4d93-a7fd-8d8f0306c65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:22.586732 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:22.606507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-70c89cd7-d926-4d93-a7fd-8d8f0306c65e None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:22.608079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-70c89cd7-d926-4d93-a7fd-8d8f0306c65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:22.621231 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1b350-f328-46d2-b68b-3a05fa989498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:22.621853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1b350-f328-46d2-b68b-3a05fa989498 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:22.622137 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1b350-f328-46d2-b68b-3a05fa989498 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:22.622410 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1b350-f328-46d2-b68b-3a05fa989498 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:22.623179 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1b350-f328-46d2-b68b-3a05fa989498 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:22.623482 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1b350-f328-46d2-b68b-3a05fa989498 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:22.628438 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1b350-f328-46d2-b68b-3a05fa989498 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:22.635241 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 162/342] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:22.636235 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1b350-f328-46d2-b68b-3a05fa989498 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:22.709504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 181/343] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:22.710857 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c89cd7-d926-4d93-a7fd-8d8f0306c65e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:22.770472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1b350-f328-46d2-b68b-3a05fa989498 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:22.804440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:22.810220 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c159435-03d4-4d92-bf96-c1fe7b98a9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:22.810863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c159435-03d4-4d92-bf96-c1fe7b98a9bf None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:22.811163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c159435-03d4-4d92-bf96-c1fe7b98a9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:22.811416 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c159435-03d4-4d92-bf96-c1fe7b98a9bf None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:22.812171 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c159435-03d4-4d92-bf96-c1fe7b98a9bf None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:22.812531 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c159435-03d4-4d92-bf96-c1fe7b98a9bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:22.813948 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c159435-03d4-4d92-bf96-c1fe7b98a9bf None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:22.823410 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 163/344] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:22.824249 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c159435-03d4-4d92-bf96-c1fe7b98a9bf None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:22.885494 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:22.903923 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c89cd7-d926-4d93-a7fd-8d8f0306c65e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:22.931790 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c159435-03d4-4d92-bf96-c1fe7b98a9bf None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:22.970552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ecbff-a128-4097-99c6-04b260451e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:22.972053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ecbff-a128-4097-99c6-04b260451e3b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:22.972539 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ecbff-a128-4097-99c6-04b260451e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:22.972950 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ecbff-a128-4097-99c6-04b260451e3b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:22.984307 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ecbff-a128-4097-99c6-04b260451e3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:22.992443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-0d2ecbff-a128-4097-99c6-04b260451e3b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:22.992443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:22.992443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:22.992443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:22.992443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:22.992443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:22.992443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:22.992443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:22.992443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:22.992443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:22.992443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:22.992443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:22.992443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:22.992443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:22.992443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:22.992443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:22.993743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:22.993743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:22.993743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:22.993743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:22.993743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:22.993743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:22.993743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:22.993743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:22.993743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:22.993743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:22.993743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:22.993743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:22.993743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:22.993743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:22.993743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:22.993743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:22.994975 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:22.994975 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:22.994975 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:22.994975 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:22.994975 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:22.994975 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:22.994975 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:22.994975 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:22.994975 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:22.994975 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:22.994975 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:22.994975 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:22.997081 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 164/345] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:22.998034 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ecbff-a128-4097-99c6-04b260451e3b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:23.003675 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26875fe1-c1fe-45b7-8780-20b741d10418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:23.004466 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26875fe1-c1fe-45b7-8780-20b741d10418 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:23.004819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26875fe1-c1fe-45b7-8780-20b741d10418 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:23.005166 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26875fe1-c1fe-45b7-8780-20b741d10418 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:23.128402 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26875fe1-c1fe-45b7-8780-20b741d10418 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:23.130331 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26875fe1-c1fe-45b7-8780-20b741d10418 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:23.166132 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ecbff-a128-4097-99c6-04b260451e3b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:23.183858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:23.217456 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c02ddd-3a96-45f6-ad7a-db72913d2c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:23.218886 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c02ddd-3a96-45f6-ad7a-db72913d2c6d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:23.219375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c02ddd-3a96-45f6-ad7a-db72913d2c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:23.219852 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c02ddd-3a96-45f6-ad7a-db72913d2c6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:23.275728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:23.310764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c02ddd-3a96-45f6-ad7a-db72913d2c6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:23.312793 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c02ddd-3a96-45f6-ad7a-db72913d2c6d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:23.341221 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 182/346] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:23.342401 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26875fe1-c1fe-45b7-8780-20b741d10418 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:23.437273 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 165/347] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:23.439017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a5784-a7e1-4581-81df-67240b7215f6 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:23.439496 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a5784-a7e1-4581-81df-67240b7215f6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:23.440001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a5784-a7e1-4581-81df-67240b7215f6 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:23.441500 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 166/348] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:23.442378 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a5784-a7e1-4581-81df-67240b7215f6 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:23.455990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:23.477683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26875fe1-c1fe-45b7-8780-20b741d10418 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:23.522986 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06b0b5-71fd-4b3e-857c-e5561aefe5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:23.523665 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06b0b5-71fd-4b3e-857c-e5561aefe5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:23.524311 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06b0b5-71fd-4b3e-857c-e5561aefe5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:23.524633 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06b0b5-71fd-4b3e-857c-e5561aefe5f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:23.525590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06b0b5-71fd-4b3e-857c-e5561aefe5f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:23.526060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06b0b5-71fd-4b3e-857c-e5561aefe5f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:23.527619 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06b0b5-71fd-4b3e-857c-e5561aefe5f0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:23.538996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 183/349] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:23.540326 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4c6c9-9d61-4243-bc89-221da5eea516 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:23.540661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4c6c9-9d61-4243-bc89-221da5eea516 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:23.540943 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c4c6c9-9d61-4243-bc89-221da5eea516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:23.563811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-70c4c6c9-9d61-4243-bc89-221da5eea516 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:23.565275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-70c4c6c9-9d61-4243-bc89-221da5eea516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:23.576554 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a5784-a7e1-4581-81df-67240b7215f6 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:23.594043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:23.624089 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac64ad-fba7-407b-b4ca-5edde4c1f3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:23.625492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac64ad-fba7-407b-b4ca-5edde4c1f3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:23.625967 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac64ad-fba7-407b-b4ca-5edde4c1f3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:23.626429 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac64ad-fba7-407b-b4ca-5edde4c1f3a9 None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:23.627976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac64ad-fba7-407b-b4ca-5edde4c1f3a9 None admin] RBAC: Authorizing `identity:get_project(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:23.629925 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac64ad-fba7-407b-b4ca-5edde4c1f3a9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:23.632016 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 167/350] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:23 2022] GET /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:23.633241 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac64ad-fba7-407b-b4ca-5edde4c1f3a9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:23.649467 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 184/351] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:23.651378 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c4c6c9-9d61-4243-bc89-221da5eea516 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:23.743832 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:23.771255 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac64ad-fba7-407b-b4ca-5edde4c1f3a9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:23.786269 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c4c6c9-9d61-4243-bc89-221da5eea516 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:23.814274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c0706-c866-4616-b925-d081fdcc61f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:23.815496 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c0706-c866-4616-b925-d081fdcc61f4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:23.816026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c0706-c866-4616-b925-d081fdcc61f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:23.816309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c0706-c866-4616-b925-d081fdcc61f4 None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:23.817822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c0706-c866-4616-b925-d081fdcc61f4 None admin] RBAC: Authorizing `identity:get_project(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:23.819846 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c0706-c866-4616-b925-d081fdcc61f4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:23.821907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 168/352] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:23 2022] GET /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:23.823004 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c0706-c866-4616-b925-d081fdcc61f4 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:23.828905 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6e058-a4dd-486a-9efc-3848a91888bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:23.829568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6e058-a4dd-486a-9efc-3848a91888bc None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:23.829844 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6e058-a4dd-486a-9efc-3848a91888bc None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:23.830114 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6e058-a4dd-486a-9efc-3848a91888bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:23.871663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:23.922622 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6e058-a4dd-486a-9efc-3848a91888bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:23.924083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6e058-a4dd-486a-9efc-3848a91888bc None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:23.953638 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c0706-c866-4616-b925-d081fdcc61f4 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:24.005668 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f6c81-cd44-4f9e-866e-b4ad747d301e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:24.007783 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f6c81-cd44-4f9e-866e-b4ad747d301e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:24.008207 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f6c81-cd44-4f9e-866e-b4ad747d301e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:24.008671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f6c81-cd44-4f9e-866e-b4ad747d301e None admin] PATH_INFO: `/v3/groups/1ebfe049096c4d839a02d5d9d829c95a` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:24.016036 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f6c81-cd44-4f9e-866e-b4ad747d301e None admin] RBAC: Authorizing `identity:get_group(group_id=1ebfe049096c4d839a02d5d9d829c95a)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:24.017894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f6c81-cd44-4f9e-866e-b4ad747d301e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:24.020084 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 169/353] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Jun 25 15:53:23 2022] GET /identity/v3/groups/1ebfe049096c4d839a02d5d9d829c95a => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:24.021561 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e2fc4-7454-4359-97a3-ca773e6d7f76 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:24.021978 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e2fc4-7454-4359-97a3-ca773e6d7f76 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:24.022374 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e2fc4-7454-4359-97a3-ca773e6d7f76 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:24.031236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 170/354] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:24.031236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e2fc4-7454-4359-97a3-ca773e6d7f76 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:24.045000 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:24.050337 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 185/355] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:24.055692 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6e058-a4dd-486a-9efc-3848a91888bc None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:24.129827 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:24.165232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e2fc4-7454-4359-97a3-ca773e6d7f76 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:24.174855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6e058-a4dd-486a-9efc-3848a91888bc None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:24.210493 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250a131-d16f-4fb1-a934-9596a60f5464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:24.211814 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250a131-d16f-4fb1-a934-9596a60f5464 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:24.212487 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250a131-d16f-4fb1-a934-9596a60f5464 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:24.212941 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250a131-d16f-4fb1-a934-9596a60f5464 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:24.214214 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42929144-f20d-4cf7-8428-8977701ca46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:24.215075 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42929144-f20d-4cf7-8428-8977701ca46e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:24.215356 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42929144-f20d-4cf7-8428-8977701ca46e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:24.215629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42929144-f20d-4cf7-8428-8977701ca46e None admin] PATH_INFO: `/v3/groups/1ebfe049096c4d839a02d5d9d829c95a` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:24.217260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42929144-f20d-4cf7-8428-8977701ca46e None admin] RBAC: Authorizing `identity:get_group(group_id=1ebfe049096c4d839a02d5d9d829c95a)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:24.218701 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42929144-f20d-4cf7-8428-8977701ca46e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:24.220563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 186/356] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Jun 25 15:53:24 2022] GET /identity/v3/groups/1ebfe049096c4d839a02d5d9d829c95a => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:24.221209 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42929144-f20d-4cf7-8428-8977701ca46e None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:24.234096 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250a131-d16f-4fb1-a934-9596a60f5464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:24.241987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-2250a131-d16f-4fb1-a934-9596a60f5464 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 25 15:53:24.241987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:24.241987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:24.241987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:24.241987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:24.241987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:24.241987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:24.241987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:24.241987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:24.241987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:24.241987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:24.241987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:24.241987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:24.241987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:24.241987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:24.241987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:24.242882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:24.242882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:24.242882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:24.242882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:24.242882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:24.242882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:24.242882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:24.242882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:24.242882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:24.242882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:24.242882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:24.242882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:24.242882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:24.242882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:24.242882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:24.242882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:24.244163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:24.244163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:24.244163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:24.244163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:24.244163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:24.244163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:24.244163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:24.244163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:24.244163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:24.244163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:24.244163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 25 15:53:24.244163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:24.244163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 171/357] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:24.246194 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333cbcc-b988-4ec0-b44b-8b368afc77bc None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:24.246640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333cbcc-b988-4ec0-b44b-8b368afc77bc None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:24.247039 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333cbcc-b988-4ec0-b44b-8b368afc77bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:24.265317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-3333cbcc-b988-4ec0-b44b-8b368afc77bc None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:24.267663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3333cbcc-b988-4ec0-b44b-8b368afc77bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:24.339696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42929144-f20d-4cf7-8428-8977701ca46e None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:24.340641 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:24.352479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 172/358] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:24.353465 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333cbcc-b988-4ec0-b44b-8b368afc77bc None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:24.373144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:24.384841 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134d6c4-d127-4fc9-8893-0bf2fcd0599c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:24.385498 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134d6c4-d127-4fc9-8893-0bf2fcd0599c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:24.385775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134d6c4-d127-4fc9-8893-0bf2fcd0599c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:24.386275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134d6c4-d127-4fc9-8893-0bf2fcd0599c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:24.387810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134d6c4-d127-4fc9-8893-0bf2fcd0599c None admin] RBAC: Adding query filter params (group.id=1ebfe049096c4d839a02d5d9d829c95a, role.id=6322fb4a438d4a389c43d05248665e72, scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:24.387810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134d6c4-d127-4fc9-8893-0bf2fcd0599c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:24.388651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134d6c4-d127-4fc9-8893-0bf2fcd0599c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:24.395796 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 187/359] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Jun 25 15:53:24 2022] GET /identity/v3/role_assignments?group.id=1ebfe049096c4d839a02d5d9d829c95a&scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea&role.id=6322fb4a438d4a389c43d05248665e72 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:24.396638 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134d6c4-d127-4fc9-8893-0bf2fcd0599c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:24.500963 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3333cbcc-b988-4ec0-b44b-8b368afc77bc None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:24.540145 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134d6c4-d127-4fc9-8893-0bf2fcd0599c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:24.547712 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbbc42-a606-41c3-b0ef-925580f6b4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:24.548663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbbc42-a606-41c3-b0ef-925580f6b4fe None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:24.548934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbbc42-a606-41c3-b0ef-925580f6b4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:24.549196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbbc42-a606-41c3-b0ef-925580f6b4fe None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:24.550438 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbbc42-a606-41c3-b0ef-925580f6b4fe None admin] RBAC: Authorizing `identity:get_project(project_id=3233ec59a0994511b0032830b4ded6c2)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:24.551900 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbbc42-a606-41c3-b0ef-925580f6b4fe None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:24.554979 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 173/360] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:24 2022] GET /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:24.554979 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbbc42-a606-41c3-b0ef-925580f6b4fe None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:24.592970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2281093-a0fe-4225-8691-0af10fb1f916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:24.593862 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2281093-a0fe-4225-8691-0af10fb1f916 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:24.594176 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2281093-a0fe-4225-8691-0af10fb1f916 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:24.595329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2281093-a0fe-4225-8691-0af10fb1f916 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:24.596189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2281093-a0fe-4225-8691-0af10fb1f916 None admin] RBAC: Adding query filter params (group.id=1ebfe049096c4d839a02d5d9d829c95a, role.id=8b9bae8f81854757b4f389d3bd22f595, scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:24.596571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2281093-a0fe-4225-8691-0af10fb1f916 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:24.598014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2281093-a0fe-4225-8691-0af10fb1f916 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:24.607277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 188/361] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Jun 25 15:53:24 2022] GET /identity/v3/role_assignments?group.id=1ebfe049096c4d839a02d5d9d829c95a&scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea&role.id=8b9bae8f81854757b4f389d3bd22f595 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:24.608833 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b952c1f-1ef2-41d3-a4b0-890d6ed129e5 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:24.609120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b952c1f-1ef2-41d3-a4b0-890d6ed129e5 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:24.609386 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b952c1f-1ef2-41d3-a4b0-890d6ed129e5 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:24.610887 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 189/362] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:24.615838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b952c1f-1ef2-41d3-a4b0-890d6ed129e5 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:24.639943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:24.690760 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:24.694374 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbbc42-a606-41c3-b0ef-925580f6b4fe None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:24.745751 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b3c01-e098-4fdf-ad8a-ed250a113b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:24.747018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b3c01-e098-4fdf-ad8a-ed250a113b01 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:24.747504 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b3c01-e098-4fdf-ad8a-ed250a113b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:24.747679 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b952c1f-1ef2-41d3-a4b0-890d6ed129e5 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:24.748147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b3c01-e098-4fdf-ad8a-ed250a113b01 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:24.749158 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b3c01-e098-4fdf-ad8a-ed250a113b01 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:24.749675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b3c01-e098-4fdf-ad8a-ed250a113b01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:24.751297 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b3c01-e098-4fdf-ad8a-ed250a113b01 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:24.761400 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 174/363] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:24.762607 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b3c01-e098-4fdf-ad8a-ed250a113b01 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:24.786954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d4ccc-1f4e-4236-9b36-b4851786807c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:24.787785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d4ccc-1f4e-4236-9b36-b4851786807c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:24.788129 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d4ccc-1f4e-4236-9b36-b4851786807c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:24.788448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d4ccc-1f4e-4236-9b36-b4851786807c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:24.802004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d4ccc-1f4e-4236-9b36-b4851786807c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:24.810493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-cd9d4ccc-1f4e-4236-9b36-b4851786807c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:24.810493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:24.810493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:24.810493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:24.810493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:24.810493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:24.810493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:24.810493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:24.810493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:24.810493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:24.810493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:24.810493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:24.810493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:24.810493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:24.810493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:24.810493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:24.813583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:24.813583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:24.813583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:24.813583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:24.813583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:24.813583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:24.813583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:24.813583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:24.813583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:24.813583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:24.813583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:24.813583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:24.813583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:24.813583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:24.813583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:24.813583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:24.814432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:24.814432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:24.814432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:24.814432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:24.814432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:24.814432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:24.814432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:24.814432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:24.814432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:24.814432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:24.814432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:24.814432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:24.814432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 190/364] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:53:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:24.814432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f6f3f-9147-4c1e-bce8-2bb3723f5025 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:24.815190 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f6f3f-9147-4c1e-bce8-2bb3723f5025 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:24.815190 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f6f3f-9147-4c1e-bce8-2bb3723f5025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:24.832931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-492f6f3f-9147-4c1e-bce8-2bb3723f5025 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:24.834139 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-492f6f3f-9147-4c1e-bce8-2bb3723f5025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:24.922346 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 191/365] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:24.922346 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f6f3f-9147-4c1e-bce8-2bb3723f5025 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:24.954354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b3c01-e098-4fdf-ad8a-ed250a113b01 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:24.996243 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:25.013588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:25.018407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef31b9a-c800-43ce-a2b1-de41a7e22f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:25.019284 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef31b9a-c800-43ce-a2b1-de41a7e22f48 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:25.019558 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef31b9a-c800-43ce-a2b1-de41a7e22f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:25.019827 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef31b9a-c800-43ce-a2b1-de41a7e22f48 None admin] PATH_INFO: `/v3/groups/1ebfe049096c4d839a02d5d9d829c95a` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:25.021172 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef31b9a-c800-43ce-a2b1-de41a7e22f48 None admin] RBAC: Authorizing `identity:get_group(group_id=1ebfe049096c4d839a02d5d9d829c95a)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:25.022638 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef31b9a-c800-43ce-a2b1-de41a7e22f48 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:25.025748 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 175/366] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Jun 25 15:53:24 2022] GET /identity/v3/groups/1ebfe049096c4d839a02d5d9d829c95a => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:25.025748 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef31b9a-c800-43ce-a2b1-de41a7e22f48 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:25.118050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f6f3f-9147-4c1e-bce8-2bb3723f5025 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:25.174125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d1a26-343c-47d6-adbd-3390724d9389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:25.175029 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d1a26-343c-47d6-adbd-3390724d9389 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:25.175306 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d1a26-343c-47d6-adbd-3390724d9389 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:25.176866 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d1a26-343c-47d6-adbd-3390724d9389 None admin] PATH_INFO: `/v3/users/426b2dff36ce4570ba8eb544c4d88fd2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:25.182014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d1a26-343c-47d6-adbd-3390724d9389 None admin] RBAC: Authorizing `identity:get_user(user_id=426b2dff36ce4570ba8eb544c4d88fd2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:25.183994 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d1a26-343c-47d6-adbd-3390724d9389 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:25.188510 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 192/367] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:53:24 2022] GET /identity/v3/users/426b2dff36ce4570ba8eb544c4d88fd2 => generated 293 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:25.190029 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d1a26-343c-47d6-adbd-3390724d9389 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:25.195727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef31b9a-c800-43ce-a2b1-de41a7e22f48 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:25.242088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119692a3-6b5b-4d68-a490-b6552eb37d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:25.242634 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119692a3-6b5b-4d68-a490-b6552eb37d24 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:25.242909 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119692a3-6b5b-4d68-a490-b6552eb37d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:25.243181 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119692a3-6b5b-4d68-a490-b6552eb37d24 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:25.244201 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119692a3-6b5b-4d68-a490-b6552eb37d24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:25.248958 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119692a3-6b5b-4d68-a490-b6552eb37d24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:25.250993 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119692a3-6b5b-4d68-a490-b6552eb37d24 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:25.262411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 176/368] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:25.263255 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119692a3-6b5b-4d68-a490-b6552eb37d24 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:25.335178 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d1a26-343c-47d6-adbd-3390724d9389 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:25.336498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:25.356728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:25.387702 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfd9bf-232e-448c-90ee-a95e29331323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:25.388721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfd9bf-232e-448c-90ee-a95e29331323 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:25.389002 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfd9bf-232e-448c-90ee-a95e29331323 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:25.389282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfd9bf-232e-448c-90ee-a95e29331323 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:25.404134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfd9bf-232e-448c-90ee-a95e29331323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:25.406261 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119692a3-6b5b-4d68-a490-b6552eb37d24 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:25.413133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-edcfd9bf-232e-448c-90ee-a95e29331323 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:53:25.413133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:25.413133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:25.413133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:25.413133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:25.413133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:25.413133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:25.413133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:25.413133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:25.413133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:25.413133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:25.413133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:25.413133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:25.413133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:25.413133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:25.413133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:25.415883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:25.415883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:25.415883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:25.415883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:25.415883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:25.415883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:25.415883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:25.415883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:25.415883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:25.415883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:25.415883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:25.415883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:25.415883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:25.415883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:25.415883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:25.415883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:25.417288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:25.417288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:25.417288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:25.417288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:25.417288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:25.417288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:25.417288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:25.417288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:25.417288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:25.417288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:25.417288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:53:25.417288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:25.420498 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 193/369] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:53:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:25.422476 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfd9bf-232e-448c-90ee-a95e29331323 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:25.465283 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5cd93-d6d2-4789-84ff-6ca35b82a5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:25.465960 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5cd93-d6d2-4789-84ff-6ca35b82a5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:25.466275 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5cd93-d6d2-4789-84ff-6ca35b82a5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:25.466545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5cd93-d6d2-4789-84ff-6ca35b82a5f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:25.467623 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af5cd93-d6d2-4789-84ff-6ca35b82a5f4 None admin] RBAC: Adding query filter params (group.id=1ebfe049096c4d839a02d5d9d829c95a, role.id=8b9bae8f81854757b4f389d3bd22f595, scope.project.id=3233ec59a0994511b0032830b4ded6c2) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:25.467937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af5cd93-d6d2-4789-84ff-6ca35b82a5f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:25.469658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af5cd93-d6d2-4789-84ff-6ca35b82a5f4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:25.479501 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 177/370] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Jun 25 15:53:25 2022] GET /identity/v3/role_assignments?group.id=1ebfe049096c4d839a02d5d9d829c95a&scope.project.id=3233ec59a0994511b0032830b4ded6c2&role.id=8b9bae8f81854757b4f389d3bd22f595 => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:25.483622 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5cd93-d6d2-4789-84ff-6ca35b82a5f4 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:25.621434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfd9bf-232e-448c-90ee-a95e29331323 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:25.679866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5cd93-d6d2-4789-84ff-6ca35b82a5f4 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:25.700623 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c707ab7-6e2d-48fe-a294-26aba57a2acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:25.701310 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c707ab7-6e2d-48fe-a294-26aba57a2acf None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:25.701592 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c707ab7-6e2d-48fe-a294-26aba57a2acf None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:25.701866 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c707ab7-6e2d-48fe-a294-26aba57a2acf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:25.702727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c707ab7-6e2d-48fe-a294-26aba57a2acf None admin] RBAC: Adding query filter params (role.id=f70856a334f7479fb00bac40f893e9f0, user.id=426b2dff36ce4570ba8eb544c4d88fd2, scope.system=all) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:25.703031 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c707ab7-6e2d-48fe-a294-26aba57a2acf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:25.704555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c707ab7-6e2d-48fe-a294-26aba57a2acf None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:25.740829 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 194/371] 10.222.0.13 () {56 vars in 1308 bytes} [Sat Jun 25 15:53:25 2022] GET /identity/v3/role_assignments?user.id=426b2dff36ce4570ba8eb544c4d88fd2&scope.system=all&role.id=f70856a334f7479fb00bac40f893e9f0 => generated 510 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:25.741791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c707ab7-6e2d-48fe-a294-26aba57a2acf None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:25.746382 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:25.768484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dd1e7-323c-459e-8cdc-1c539b0146b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:25.769576 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dd1e7-323c-459e-8cdc-1c539b0146b3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:25.770073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dd1e7-323c-459e-8cdc-1c539b0146b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:25.770541 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dd1e7-323c-459e-8cdc-1c539b0146b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:25.812348 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:25.867108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95dd1e7-323c-459e-8cdc-1c539b0146b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:25.868981 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95dd1e7-323c-459e-8cdc-1c539b0146b3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:25.924435 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c707ab7-6e2d-48fe-a294-26aba57a2acf None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:25.966483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21083ff7-94bc-4ea1-95c2-1f315c40cdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:25.967360 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21083ff7-94bc-4ea1-95c2-1f315c40cdc3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:25.967849 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21083ff7-94bc-4ea1-95c2-1f315c40cdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:25.968221 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21083ff7-94bc-4ea1-95c2-1f315c40cdc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:25.972113 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21083ff7-94bc-4ea1-95c2-1f315c40cdc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:25.972113 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21083ff7-94bc-4ea1-95c2-1f315c40cdc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:25.972113 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21083ff7-94bc-4ea1-95c2-1f315c40cdc3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:25.992866 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 195/372] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:25.997468 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21083ff7-94bc-4ea1-95c2-1f315c40cdc3 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:26.079810 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 178/373] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:26.085299 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dd1e7-323c-459e-8cdc-1c539b0146b3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:26.135259 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:26.205703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:26.226472 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21083ff7-94bc-4ea1-95c2-1f315c40cdc3 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:26.284966 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dd1e7-323c-459e-8cdc-1c539b0146b3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:26.309406 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822db98-9105-47a6-9244-923252a9abcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:26.310068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822db98-9105-47a6-9244-923252a9abcf None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:26.310390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822db98-9105-47a6-9244-923252a9abcf None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:26.310665 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822db98-9105-47a6-9244-923252a9abcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:26.392601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822db98-9105-47a6-9244-923252a9abcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:26.394568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822db98-9105-47a6-9244-923252a9abcf None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:26.397110 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001c2db-7e5e-42da-a217-1976f75f4257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:26.398486 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001c2db-7e5e-42da-a217-1976f75f4257 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:26.398486 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001c2db-7e5e-42da-a217-1976f75f4257 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:26.398486 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001c2db-7e5e-42da-a217-1976f75f4257 None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:26.399807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001c2db-7e5e-42da-a217-1976f75f4257 None admin] RBAC: Authorizing `identity:get_project(project_id=3233ec59a0994511b0032830b4ded6c2)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:26.401337 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001c2db-7e5e-42da-a217-1976f75f4257 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:26.403097 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 179/374] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:26 2022] GET /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2 => generated 302 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:26.408496 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001c2db-7e5e-42da-a217-1976f75f4257 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:26.500899 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:26.514465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 196/375] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 25 15:53:26.523154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822db98-9105-47a6-9244-923252a9abcf None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:26.608596 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:26.627696 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001c2db-7e5e-42da-a217-1976f75f4257 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:26.703746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822db98-9105-47a6-9244-923252a9abcf None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:26.743064 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ecf97-4939-4e65-ae48-528bdb2d435c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:26.744564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ecf97-4939-4e65-ae48-528bdb2d435c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:26.745067 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ecf97-4939-4e65-ae48-528bdb2d435c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:26.745169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc0f7a-de34-44eb-82e3-f9dba21a516c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:26.745683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ecf97-4939-4e65-ae48-528bdb2d435c None admin] PATH_INFO: `/v3/projects/c8597e0eb76849bdadd0530f7e198ab0` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:26.745985 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc0f7a-de34-44eb-82e3-f9dba21a516c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:26.746261 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc0f7a-de34-44eb-82e3-f9dba21a516c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:26.746531 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc0f7a-de34-44eb-82e3-f9dba21a516c None admin] PATH_INFO: `/v3/groups/1ebfe049096c4d839a02d5d9d829c95a` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:26.747755 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ecf97-4939-4e65-ae48-528bdb2d435c None admin] RBAC: Authorizing `identity:get_project(project_id=c8597e0eb76849bdadd0530f7e198ab0)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:26.749441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ecf97-4939-4e65-ae48-528bdb2d435c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:26.751775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 197/376] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:26 2022] GET /identity/v3/projects/c8597e0eb76849bdadd0530f7e198ab0 => generated 344 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:26.755414 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc0f7a-de34-44eb-82e3-f9dba21a516c None admin] RBAC: Authorizing `identity:get_group(group_id=1ebfe049096c4d839a02d5d9d829c95a)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:26.755414 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc0f7a-de34-44eb-82e3-f9dba21a516c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:26.757410 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 180/377] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Jun 25 15:53:26 2022] GET /identity/v3/groups/1ebfe049096c4d839a02d5d9d829c95a => generated 226 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:26.757809 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ecf97-4939-4e65-ae48-528bdb2d435c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:26.764673 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc0f7a-de34-44eb-82e3-f9dba21a516c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:26.826832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:26.890997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ecf97-4939-4e65-ae48-528bdb2d435c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:26.927805 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc0f7a-de34-44eb-82e3-f9dba21a516c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:26.931178 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6f422-4c5a-4d02-ae71-8f42bdbf0907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:26.932127 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6f422-4c5a-4d02-ae71-8f42bdbf0907 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:26.932421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6f422-4c5a-4d02-ae71-8f42bdbf0907 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:26.932707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6f422-4c5a-4d02-ae71-8f42bdbf0907 None admin] PATH_INFO: `/v3/groups/a862c3e46e704a2183ff2995a4039670` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:26.942139 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6f422-4c5a-4d02-ae71-8f42bdbf0907 None admin] RBAC: Authorizing `identity:get_group(group_id=a862c3e46e704a2183ff2995a4039670)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:26.943588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6f422-4c5a-4d02-ae71-8f42bdbf0907 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:26.945581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 198/378] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Jun 25 15:53:26 2022] GET /identity/v3/groups/a862c3e46e704a2183ff2995a4039670 => generated 229 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:26.947193 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa99f53-f04e-4d9b-a67b-43fa08cb8ec3 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:26.947428 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa99f53-f04e-4d9b-a67b-43fa08cb8ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:26.947681 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa99f53-f04e-4d9b-a67b-43fa08cb8ec3 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:26.954855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 199/379] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:26.955628 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa99f53-f04e-4d9b-a67b-43fa08cb8ec3 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:26.979745 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017d8d8-6941-4b26-a7c0-edf960c067fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:26.980686 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017d8d8-6941-4b26-a7c0-edf960c067fc None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:26.981354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017d8d8-6941-4b26-a7c0-edf960c067fc None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:26.981633 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017d8d8-6941-4b26-a7c0-edf960c067fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:26.982474 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017d8d8-6941-4b26-a7c0-edf960c067fc None admin] RBAC: Adding query filter params (group.id=1ebfe049096c4d839a02d5d9d829c95a, role.id=6322fb4a438d4a389c43d05248665e72, scope.project.id=3233ec59a0994511b0032830b4ded6c2) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:26.982785 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017d8d8-6941-4b26-a7c0-edf960c067fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:26.984808 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017d8d8-6941-4b26-a7c0-edf960c067fc None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:26.994993 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 181/380] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Jun 25 15:53:26 2022] GET /identity/v3/role_assignments?group.id=1ebfe049096c4d839a02d5d9d829c95a&scope.project.id=3233ec59a0994511b0032830b4ded6c2&role.id=6322fb4a438d4a389c43d05248665e72 => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:26.997290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14df6a4-12ed-4d22-a5e7-83ab607ff4be None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:26.998013 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14df6a4-12ed-4d22-a5e7-83ab607ff4be None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:26.998439 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14df6a4-12ed-4d22-a5e7-83ab607ff4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:27.026776 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a14df6a4-12ed-4d22-a5e7-83ab607ff4be None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:27.030017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a14df6a4-12ed-4d22-a5e7-83ab607ff4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:27.068202 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:27.159812 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa99f53-f04e-4d9b-a67b-43fa08cb8ec3 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:27.191812 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 182/381] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:27.193212 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8edfab2-72d9-40be-9346-7035342862be None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:27.193495 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8edfab2-72d9-40be-9346-7035342862be None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:27.193767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8edfab2-72d9-40be-9346-7035342862be None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:27.195183 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 183/382] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:27.206370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:27.214388 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fb166-7a2e-4f88-8454-a976798bfd9c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:27.214664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fb166-7a2e-4f88-8454-a976798bfd9c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:27.214938 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fb166-7a2e-4f88-8454-a976798bfd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:27.229200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4455b4-e26b-4543-92c1-c28cd0ba9a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:27.229871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4455b4-e26b-4543-92c1-c28cd0ba9a46 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:27.230149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4455b4-e26b-4543-92c1-c28cd0ba9a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:27.230425 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4455b4-e26b-4543-92c1-c28cd0ba9a46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:27.231274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4455b4-e26b-4543-92c1-c28cd0ba9a46 None admin] RBAC: Adding query filter params (group.id=a862c3e46e704a2183ff2995a4039670, role.id=b1e13c449fa64b3e88b6832bb767214b, scope.project.id=c8597e0eb76849bdadd0530f7e198ab0) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:27.231579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4455b4-e26b-4543-92c1-c28cd0ba9a46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:27.233039 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4455b4-e26b-4543-92c1-c28cd0ba9a46 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:27.235212 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-504fb166-7a2e-4f88-8454-a976798bfd9c None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:27.236757 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-504fb166-7a2e-4f88-8454-a976798bfd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:27.242455 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 200/383] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Jun 25 15:53:26 2022] GET /identity/v3/role_assignments?group.id=a862c3e46e704a2183ff2995a4039670&scope.project.id=c8597e0eb76849bdadd0530f7e198ab0&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 260 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:27.242455 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4455b4-e26b-4543-92c1-c28cd0ba9a46 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:27.314653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 184/384] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 25 15:53:27.337985 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504fb166-7a2e-4f88-8454-a976798bfd9c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:27.347315 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4455b4-e26b-4543-92c1-c28cd0ba9a46 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:27.383880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7b9ed-1d68-4733-b969-0f92ec18aa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:27.385168 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7b9ed-1d68-4733-b969-0f92ec18aa9a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:27.385451 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7b9ed-1d68-4733-b969-0f92ec18aa9a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:27.385728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7b9ed-1d68-4733-b969-0f92ec18aa9a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:27.396954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7b9ed-1d68-4733-b969-0f92ec18aa9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:27.405513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-5ed7b9ed-1d68-4733-b969-0f92ec18aa9a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:27.405513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:27.405513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:27.405513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:27.405513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:27.405513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:27.405513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:27.405513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:27.405513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:27.405513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:27.405513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:27.405513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:27.405513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:27.405513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:27.405513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:27.405513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:27.406477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:27.406477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:27.406477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:27.406477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:27.406477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:27.406477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:27.406477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:27.406477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:27.406477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:27.406477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:27.406477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:27.406477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:27.406477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:27.406477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:27.406477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:27.406477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:27.407253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:27.407253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:27.407253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:27.407253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:27.407253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:27.407253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:27.407253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:27.407253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:27.407253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:27.407253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:27.407253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:27.407253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:27.410578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 201/385] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:53:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:27.413597 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7b9ed-1d68-4733-b969-0f92ec18aa9a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:27.429403 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:27.451265 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504fb166-7a2e-4f88-8454-a976798bfd9c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:27.472866 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:27.492375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82deb8-f032-42af-bf1e-52928d6fc63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:27.492991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82deb8-f032-42af-bf1e-52928d6fc63e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:27.493272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82deb8-f032-42af-bf1e-52928d6fc63e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:27.493564 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82deb8-f032-42af-bf1e-52928d6fc63e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:27.505944 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82deb8-f032-42af-bf1e-52928d6fc63e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:27.514070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-1e82deb8-f032-42af-bf1e-52928d6fc63e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:27.514070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:27.514070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:27.514070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:27.514070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:27.514070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:27.514070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:27.514070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:27.514070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:27.514070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:27.514070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:27.514070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:27.514070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:27.514070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:27.514070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:27.514070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:27.515050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:27.515050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:27.515050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:27.515050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:27.515050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:27.515050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:27.515050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:27.515050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:27.515050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:27.515050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:27.515050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:27.515050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:27.515050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:27.515050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:27.515050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:27.515050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:27.515829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:27.515829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:27.515829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:27.515829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:27.515829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:27.515829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:27.515829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:27.515829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:27.515829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:27.515829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:27.515829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:27.515829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:27.515829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 185/386] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:27.522959 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82deb8-f032-42af-bf1e-52928d6fc63e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:27.536700 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7b9ed-1d68-4733-b969-0f92ec18aa9a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:27.576575 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c1042-99d6-41bd-a5c9-a7d5b227c3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:27.577832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c1042-99d6-41bd-a5c9-a7d5b227c3ae None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:27.578407 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c1042-99d6-41bd-a5c9-a7d5b227c3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:27.578869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c1042-99d6-41bd-a5c9-a7d5b227c3ae None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:27.579817 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c1042-99d6-41bd-a5c9-a7d5b227c3ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:27.580376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c1042-99d6-41bd-a5c9-a7d5b227c3ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:27.582572 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c1042-99d6-41bd-a5c9-a7d5b227c3ae None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:27.593856 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 202/387] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:27.599790 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c1042-99d6-41bd-a5c9-a7d5b227c3ae None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:27.635977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82deb8-f032-42af-bf1e-52928d6fc63e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:27.670855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7619e-3d26-4698-96cd-7865725b6820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:27.671451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7619e-3d26-4698-96cd-7865725b6820 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:27.671729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7619e-3d26-4698-96cd-7865725b6820 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:27.672002 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7619e-3d26-4698-96cd-7865725b6820 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:27.672790 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7619e-3d26-4698-96cd-7865725b6820 None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:27.673085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7619e-3d26-4698-96cd-7865725b6820 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:27.674470 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7619e-3d26-4698-96cd-7865725b6820 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:27.688788 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 186/388] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:27.689689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7619e-3d26-4698-96cd-7865725b6820 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:27.693389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:27.717779 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c1042-99d6-41bd-a5c9-a7d5b227c3ae None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:27.736795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:27.760926 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f2599-b81e-486d-b69a-09dc87ccf348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:27.762249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f2599-b81e-486d-b69a-09dc87ccf348 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:27.762757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f2599-b81e-486d-b69a-09dc87ccf348 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:27.763220 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f2599-b81e-486d-b69a-09dc87ccf348 None admin] PATH_INFO: `/v3/groups/1ebfe049096c4d839a02d5d9d829c95a` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:27.768901 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f2599-b81e-486d-b69a-09dc87ccf348 None admin] RBAC: Authorizing `identity:get_group(group_id=1ebfe049096c4d839a02d5d9d829c95a)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:27.770668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f2599-b81e-486d-b69a-09dc87ccf348 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:27.772738 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 203/389] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Jun 25 15:53:27 2022] GET /identity/v3/groups/1ebfe049096c4d839a02d5d9d829c95a => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:27.778985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f2599-b81e-486d-b69a-09dc87ccf348 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:27.799130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7619e-3d26-4698-96cd-7865725b6820 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:27.835647 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de60ca2-859d-4709-ae2e-8518ad7e0dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:27.844463 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de60ca2-859d-4709-ae2e-8518ad7e0dfb None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:27.844463 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de60ca2-859d-4709-ae2e-8518ad7e0dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:27.844463 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de60ca2-859d-4709-ae2e-8518ad7e0dfb None admin] PATH_INFO: `/v3/groups/1ebfe049096c4d839a02d5d9d829c95a` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:27.846708 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de60ca2-859d-4709-ae2e-8518ad7e0dfb None admin] RBAC: Authorizing `identity:get_group(group_id=1ebfe049096c4d839a02d5d9d829c95a)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:27.849799 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de60ca2-859d-4709-ae2e-8518ad7e0dfb None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:27.851847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 187/390] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Jun 25 15:53:27 2022] GET /identity/v3/groups/1ebfe049096c4d839a02d5d9d829c95a => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:27.858796 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de60ca2-859d-4709-ae2e-8518ad7e0dfb None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:27.977742 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:27.982436 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f2599-b81e-486d-b69a-09dc87ccf348 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:28.005297 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de60ca2-859d-4709-ae2e-8518ad7e0dfb None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:28.045344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ad9b6-c7c4-400a-8ed1-52e23159dcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:28.046276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ad9b6-c7c4-400a-8ed1-52e23159dcba None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:28.046557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ad9b6-c7c4-400a-8ed1-52e23159dcba None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:28.046874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ad9b6-c7c4-400a-8ed1-52e23159dcba None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:28.048851 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ad9b6-c7c4-400a-8ed1-52e23159dcba None admin] RBAC: Authorizing `identity:get_project(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:28.050494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ad9b6-c7c4-400a-8ed1-52e23159dcba None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:28.052838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 204/391] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:27 2022] GET /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:28.058922 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ad9b6-c7c4-400a-8ed1-52e23159dcba None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:28.092896 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b4584-2c0b-45c1-9866-2d195cbced6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:28.093687 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b4584-2c0b-45c1-9866-2d195cbced6c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:28.093960 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b4584-2c0b-45c1-9866-2d195cbced6c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:28.094228 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b4584-2c0b-45c1-9866-2d195cbced6c None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:28.096453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b4584-2c0b-45c1-9866-2d195cbced6c None admin] RBAC: Authorizing `identity:get_project(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:28.097920 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b4584-2c0b-45c1-9866-2d195cbced6c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:28.101464 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 188/392] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:27 2022] GET /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:28.106857 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b4584-2c0b-45c1-9866-2d195cbced6c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:28.141942 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:28.228608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ad9b6-c7c4-400a-8ed1-52e23159dcba None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:28.294871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fffdf3b-6249-47d6-bb27-350adb7e3a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:28.295760 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffdf3b-6249-47d6-bb27-350adb7e3a15 None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:28.296036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffdf3b-6249-47d6-bb27-350adb7e3a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:28.296309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffdf3b-6249-47d6-bb27-350adb7e3a15 None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea/groups/1ebfe049096c4d839a02d5d9d829c95a/roles/6322fb4a438d4a389c43d05248665e72` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:28.301287 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fffdf3b-6249-47d6-bb27-350adb7e3a15 None admin] RBAC: Authorizing `identity:create_grant(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea, group_id=1ebfe049096c4d839a02d5d9d829c95a, role_id=6322fb4a438d4a389c43d05248665e72)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:28.302889 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fffdf3b-6249-47d6-bb27-350adb7e3a15 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:28.303915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b4584-2c0b-45c1-9866-2d195cbced6c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:28.319549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 205/393] 10.222.0.13 () {58 vars in 1442 bytes} [Sat Jun 25 15:53:28 2022] PUT /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea/groups/1ebfe049096c4d839a02d5d9d829c95a/roles/6322fb4a438d4a389c43d05248665e72 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 15:53:28.321882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c3bdd-e744-4fe7-9c22-4eecd43e9529 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:28.322307 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c3bdd-e744-4fe7-9c22-4eecd43e9529 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:28.322486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c3bdd-e744-4fe7-9c22-4eecd43e9529 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:28.324568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 206/394] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:28.330398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0823034-2c0f-4931-890a-8d854c1c7b8f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:28.330784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0823034-2c0f-4931-890a-8d854c1c7b8f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:28.331138 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0823034-2c0f-4931-890a-8d854c1c7b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:28.347750 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4849d5b-8573-4bdf-9356-8fee25d2b963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:28.349090 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4849d5b-8573-4bdf-9356-8fee25d2b963 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:28.349564 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4849d5b-8573-4bdf-9356-8fee25d2b963 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:28.350063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4849d5b-8573-4bdf-9356-8fee25d2b963 None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea/groups/1ebfe049096c4d839a02d5d9d829c95a/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:28.352442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4849d5b-8573-4bdf-9356-8fee25d2b963 None admin] RBAC: Authorizing `identity:create_grant(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea, group_id=1ebfe049096c4d839a02d5d9d829c95a, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:28.353140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a0823034-2c0f-4931-890a-8d854c1c7b8f None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:28.354561 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4849d5b-8573-4bdf-9356-8fee25d2b963 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:28.354679 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a0823034-2c0f-4931-890a-8d854c1c7b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:28.377002 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:28.383212 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 189/395] 10.222.0.13 () {58 vars in 1442 bytes} [Sat Jun 25 15:53:28 2022] PUT /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea/groups/1ebfe049096c4d839a02d5d9d829c95a/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:53:28.438039 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 207/396] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:28.473332 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4849d5b-8573-4bdf-9356-8fee25d2b963 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:28.611483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4849d5b-8573-4bdf-9356-8fee25d2b963 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:28.652616 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220db032-9256-4d47-ba30-4107474f8906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:28.653454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220db032-9256-4d47-ba30-4107474f8906 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:28.653737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220db032-9256-4d47-ba30-4107474f8906 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:28.654015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220db032-9256-4d47-ba30-4107474f8906 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:28.664622 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220db032-9256-4d47-ba30-4107474f8906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:28.672604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-220db032-9256-4d47-ba30-4107474f8906 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:28.672604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:28.672604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:28.672604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:28.672604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:28.672604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:28.672604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:28.672604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:28.672604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:28.672604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:28.672604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:28.672604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:28.672604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:28.672604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:28.672604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:28.672604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:28.673525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:28.673525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:28.673525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:28.673525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:28.673525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:28.673525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:28.673525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:28.673525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:28.673525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:28.673525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:28.673525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:28.673525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:28.673525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:28.673525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:28.673525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:28.673525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:28.674557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:28.674557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:28.674557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:28.674557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:28.674557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:28.674557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:28.674557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:28.674557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:28.674557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:28.674557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:28.674557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:28.674557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:28.674557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 190/397] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:28.680022 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0823034-2c0f-4931-890a-8d854c1c7b8f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:28.736799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:28.817775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0823034-2c0f-4931-890a-8d854c1c7b8f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:28.870587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa8491-bdbf-4ed6-b07a-3e1392db86c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:28.871218 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa8491-bdbf-4ed6-b07a-3e1392db86c4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:28.871499 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa8491-bdbf-4ed6-b07a-3e1392db86c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:28.871769 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa8491-bdbf-4ed6-b07a-3e1392db86c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:28.872560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa8491-bdbf-4ed6-b07a-3e1392db86c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:28.872867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa8491-bdbf-4ed6-b07a-3e1392db86c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:28.874242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa8491-bdbf-4ed6-b07a-3e1392db86c4 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:28.887955 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 208/398] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:28.893500 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220db032-9256-4d47-ba30-4107474f8906 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:28.977399 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:29.019250 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220db032-9256-4d47-ba30-4107474f8906 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:29.059916 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06482c-d9ba-4588-ad0a-279fc03cc2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:29.060805 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06482c-d9ba-4588-ad0a-279fc03cc2ec None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:29.061083 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06482c-d9ba-4588-ad0a-279fc03cc2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:29.061351 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06482c-d9ba-4588-ad0a-279fc03cc2ec None admin] PATH_INFO: `/v3/groups/1ebfe049096c4d839a02d5d9d829c95a` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:29.062542 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06482c-d9ba-4588-ad0a-279fc03cc2ec None admin] RBAC: Authorizing `identity:get_group(group_id=1ebfe049096c4d839a02d5d9d829c95a)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:29.063992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06482c-d9ba-4588-ad0a-279fc03cc2ec None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:29.065984 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 191/399] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Jun 25 15:53:28 2022] GET /identity/v3/groups/1ebfe049096c4d839a02d5d9d829c95a => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:29.071212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa8491-bdbf-4ed6-b07a-3e1392db86c4 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:29.195758 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:29.200962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa8491-bdbf-4ed6-b07a-3e1392db86c4 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:29.247387 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e916c-6d73-4af9-ab73-e9ee55cce5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:29.248308 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e916c-6d73-4af9-ab73-e9ee55cce5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:29.248617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e916c-6d73-4af9-ab73-e9ee55cce5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:29.248908 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e916c-6d73-4af9-ab73-e9ee55cce5f1 None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:29.253170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e916c-6d73-4af9-ab73-e9ee55cce5f1 None admin] RBAC: Authorizing `identity:get_project(project_id=3233ec59a0994511b0032830b4ded6c2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:29.254625 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e916c-6d73-4af9-ab73-e9ee55cce5f1 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:29.256709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 209/400] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:29 2022] GET /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:29.262353 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06482c-d9ba-4588-ad0a-279fc03cc2ec None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:29.381678 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06482c-d9ba-4588-ad0a-279fc03cc2ec None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:29.408694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:29.425298 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8dc58b-e4ca-4179-a00b-86372c78e44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:29.426194 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dc58b-e4ca-4179-a00b-86372c78e44a None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:29.426470 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dc58b-e4ca-4179-a00b-86372c78e44a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:29.426741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dc58b-e4ca-4179-a00b-86372c78e44a None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2/groups/1ebfe049096c4d839a02d5d9d829c95a/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:29.428744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8dc58b-e4ca-4179-a00b-86372c78e44a None admin] RBAC: Authorizing `identity:create_grant(project_id=3233ec59a0994511b0032830b4ded6c2, group_id=1ebfe049096c4d839a02d5d9d829c95a, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:29.430189 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8dc58b-e4ca-4179-a00b-86372c78e44a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:29.452571 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 192/401] 10.222.0.13 () {58 vars in 1442 bytes} [Sat Jun 25 15:53:29 2022] PUT /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2/groups/1ebfe049096c4d839a02d5d9d829c95a/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 15:53:29.666044 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f54078-5903-4059-890f-f181d5158db4 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:29.666741 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f54078-5903-4059-890f-f181d5158db4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:29.667221 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f54078-5903-4059-890f-f181d5158db4 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:29.669086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 210/402] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:29.674429 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa4d9d-b922-4a8a-a187-13e4aeb82887 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:29.674728 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa4d9d-b922-4a8a-a187-13e4aeb82887 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:29.674984 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa4d9d-b922-4a8a-a187-13e4aeb82887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:29.701597 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-72fa4d9d-b922-4a8a-a187-13e4aeb82887 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:29.702898 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-72fa4d9d-b922-4a8a-a187-13e4aeb82887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:29.774583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 193/403] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:53:29.797881 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f54078-5903-4059-890f-f181d5158db4 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:29.909745 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f54078-5903-4059-890f-f181d5158db4 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:29.944418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9e883-5b85-44b7-9600-c6a254aade9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:29.945229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9e883-5b85-44b7-9600-c6a254aade9e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:29.945509 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9e883-5b85-44b7-9600-c6a254aade9e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:29.945783 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9e883-5b85-44b7-9600-c6a254aade9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:29.956434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b9e883-5b85-44b7-9600-c6a254aade9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:29.964088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-b0b9e883-5b85-44b7-9600-c6a254aade9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:53:29.964088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:29.964088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:29.964088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:29.964088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:29.964088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:29.964088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:29.964088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:29.964088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:29.964088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:29.964088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:29.964088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:29.964088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:29.964088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:29.964088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:29.964088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:29.964995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:29.964995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:29.964995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:29.964995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:29.964995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:29.964995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:29.964995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:29.964995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:29.964995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:29.964995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:29.964995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:29.964995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:29.964995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:29.964995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:29.964995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:29.964995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:29.965771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:29.965771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:29.965771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:29.965771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:29.965771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:29.965771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:29.965771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:29.965771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:29.965771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:29.965771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:29.965771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:53:29.965771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:29.965771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 211/404] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:53:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:29.971614 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa4d9d-b922-4a8a-a187-13e4aeb82887 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:30.052628 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3b27c-19ee-4ac9-b010-a323b9725e8f None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:30.052913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3b27c-19ee-4ac9-b010-a323b9725e8f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:30.053204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3b27c-19ee-4ac9-b010-a323b9725e8f None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:30.054589 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 212/405] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:30.061497 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fd6e3-3654-4b9a-bd18-05ad770f3dd5 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:30.061777 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fd6e3-3654-4b9a-bd18-05ad770f3dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:30.062053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fd6e3-3654-4b9a-bd18-05ad770f3dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:30.086707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ff2fd6e3-3654-4b9a-bd18-05ad770f3dd5 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:30.088027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ff2fd6e3-3654-4b9a-bd18-05ad770f3dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:30.114061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:30.155075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa4d9d-b922-4a8a-a187-13e4aeb82887 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:30.191534 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 213/406] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:53:30.192036 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:30.212886 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c155c27-1b15-48ce-8dd8-5ee6c6e7b80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:30.213572 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c155c27-1b15-48ce-8dd8-5ee6c6e7b80a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:30.213878 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c155c27-1b15-48ce-8dd8-5ee6c6e7b80a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:30.214152 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c155c27-1b15-48ce-8dd8-5ee6c6e7b80a None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:30.214931 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c155c27-1b15-48ce-8dd8-5ee6c6e7b80a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:30.215415 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c155c27-1b15-48ce-8dd8-5ee6c6e7b80a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:30.216311 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fd6e3-3654-4b9a-bd18-05ad770f3dd5 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:30.230088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c155c27-1b15-48ce-8dd8-5ee6c6e7b80a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:30.239545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 194/407] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:30.245028 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c155c27-1b15-48ce-8dd8-5ee6c6e7b80a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:30.349777 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c155c27-1b15-48ce-8dd8-5ee6c6e7b80a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:30.385309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343be57-4a57-4b4e-aa32-f59607d2311d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:30.386184 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343be57-4a57-4b4e-aa32-f59607d2311d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:30.386463 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343be57-4a57-4b4e-aa32-f59607d2311d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:30.386736 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343be57-4a57-4b4e-aa32-f59607d2311d None admin] PATH_INFO: `/v3/groups/a862c3e46e704a2183ff2995a4039670` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:30.387923 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343be57-4a57-4b4e-aa32-f59607d2311d None admin] RBAC: Authorizing `identity:get_group(group_id=a862c3e46e704a2183ff2995a4039670)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:30.389470 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343be57-4a57-4b4e-aa32-f59607d2311d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:30.390629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fd6e3-3654-4b9a-bd18-05ad770f3dd5 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:30.391808 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 195/408] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Jun 25 15:53:30 2022] GET /identity/v3/groups/a862c3e46e704a2183ff2995a4039670 => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:30.397165 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343be57-4a57-4b4e-aa32-f59607d2311d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:30.453180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:30.461900 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e58cd-56eb-4805-a416-f3e2a888201a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:30.462830 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e58cd-56eb-4805-a416-f3e2a888201a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:30.463166 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e58cd-56eb-4805-a416-f3e2a888201a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:30.463390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e58cd-56eb-4805-a416-f3e2a888201a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:30.497409 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e58cd-56eb-4805-a416-f3e2a888201a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:30.501682 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343be57-4a57-4b4e-aa32-f59607d2311d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:30.512151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-c06e58cd-56eb-4805-a416-f3e2a888201a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:30.512151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:30.512151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:30.512151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:30.512151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:30.512151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:30.512151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:30.512151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:30.512151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:30.512151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:30.512151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:30.512151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:30.512151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:30.512151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:30.512151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:30.512151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:30.513083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:30.513083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:30.513083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:30.513083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:30.513083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:30.513083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:30.513083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:30.513083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:30.513083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:30.513083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:30.513083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:30.513083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:30.513083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:30.513083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:30.513083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:30.513083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:30.513867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:30.513867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:30.513867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:30.513867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:30.513867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:30.513867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:30.513867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:30.513867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:30.513867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:30.513867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:30.513867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:30.513867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:30.518702 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 214/409] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:53:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:30.525725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e58cd-56eb-4805-a416-f3e2a888201a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:30.536920 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc3fba-480d-4d7c-a860-5b0dbdd072fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:30.537784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc3fba-480d-4d7c-a860-5b0dbdd072fa None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:30.538062 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc3fba-480d-4d7c-a860-5b0dbdd072fa None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:30.538331 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc3fba-480d-4d7c-a860-5b0dbdd072fa None admin] PATH_INFO: `/v3/projects/c8597e0eb76849bdadd0530f7e198ab0` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:30.539601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc3fba-480d-4d7c-a860-5b0dbdd072fa None admin] RBAC: Authorizing `identity:get_project(project_id=c8597e0eb76849bdadd0530f7e198ab0)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:30.541532 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc3fba-480d-4d7c-a860-5b0dbdd072fa None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:30.543618 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 196/410] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:30 2022] GET /identity/v3/projects/c8597e0eb76849bdadd0530f7e198ab0 => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:30.549232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc3fba-480d-4d7c-a860-5b0dbdd072fa None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:30.567879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:30.689908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc3fba-480d-4d7c-a860-5b0dbdd072fa None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:30.707976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:30.728509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c91106-10e2-4aff-aecc-21298f668783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:30.730023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c91106-10e2-4aff-aecc-21298f668783 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:30.730517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c91106-10e2-4aff-aecc-21298f668783 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:30.730979 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c91106-10e2-4aff-aecc-21298f668783 None admin] PATH_INFO: `/v3/projects/c8597e0eb76849bdadd0530f7e198ab0/groups/a862c3e46e704a2183ff2995a4039670/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:30.733388 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c91106-10e2-4aff-aecc-21298f668783 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8597e0eb76849bdadd0530f7e198ab0, group_id=a862c3e46e704a2183ff2995a4039670, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:30.735086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c91106-10e2-4aff-aecc-21298f668783 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:30.752638 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 197/411] 10.222.0.13 () {58 vars in 1442 bytes} [Sat Jun 25 15:53:30 2022] PUT /identity/v3/projects/c8597e0eb76849bdadd0530f7e198ab0/groups/a862c3e46e704a2183ff2995a4039670/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 15:53:30.796941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e58cd-56eb-4805-a416-f3e2a888201a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:30.832187 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafb25d-fc87-4440-a430-6f8c152bf149 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:30.833452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafb25d-fc87-4440-a430-6f8c152bf149 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:30.833923 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafb25d-fc87-4440-a430-6f8c152bf149 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:30.847473 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 198/412] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:30.847473 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a3e8b-51ab-4f43-8044-c504bc8e8c60 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:30.847473 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a3e8b-51ab-4f43-8044-c504bc8e8c60 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:30.847473 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a3e8b-51ab-4f43-8044-c504bc8e8c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:30.862148 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3dc7fb-6ad3-453e-8140-1b1fed624565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:30.862704 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3dc7fb-6ad3-453e-8140-1b1fed624565 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:30.863420 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3dc7fb-6ad3-453e-8140-1b1fed624565 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:30.863420 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3dc7fb-6ad3-453e-8140-1b1fed624565 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:30.869393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3dc7fb-6ad3-453e-8140-1b1fed624565 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:30.869716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3dc7fb-6ad3-453e-8140-1b1fed624565 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:30.871108 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3dc7fb-6ad3-453e-8140-1b1fed624565 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:30.875068 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-428a3e8b-51ab-4f43-8044-c504bc8e8c60 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:30.876291 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-428a3e8b-51ab-4f43-8044-c504bc8e8c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:30.898656 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 215/413] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:30.905887 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3dc7fb-6ad3-453e-8140-1b1fed624565 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:30.982758 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 199/414] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:31.010141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a3e8b-51ab-4f43-8044-c504bc8e8c60 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:31.084798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:31.096985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3dc7fb-6ad3-453e-8140-1b1fed624565 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:31.132517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a3e8b-51ab-4f43-8044-c504bc8e8c60 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:31.141785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c160c3-ad2b-4ba9-90ac-6f6953d2de01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:31.142954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:31.144031 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c160c3-ad2b-4ba9-90ac-6f6953d2de01 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:31.144320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c160c3-ad2b-4ba9-90ac-6f6953d2de01 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:31.144629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c160c3-ad2b-4ba9-90ac-6f6953d2de01 None admin] PATH_INFO: `/v3/groups/1ebfe049096c4d839a02d5d9d829c95a` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:31.146391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c160c3-ad2b-4ba9-90ac-6f6953d2de01 None admin] RBAC: Authorizing `identity:get_group(group_id=1ebfe049096c4d839a02d5d9d829c95a)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:31.148067 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c160c3-ad2b-4ba9-90ac-6f6953d2de01 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:31.156486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 216/415] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Jun 25 15:53:30 2022] GET /identity/v3/groups/1ebfe049096c4d839a02d5d9d829c95a => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:31.158129 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613e4a4-4429-45f3-a7b9-be7cd55e5c9c None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:31.158411 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613e4a4-4429-45f3-a7b9-be7cd55e5c9c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:31.158685 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613e4a4-4429-45f3-a7b9-be7cd55e5c9c None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:31.160151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 217/416] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:31.161441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613e4a4-4429-45f3-a7b9-be7cd55e5c9c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:31.189032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bab765-b44b-4c04-9efb-4f8a2055aa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:31.189830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bab765-b44b-4c04-9efb-4f8a2055aa99 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:31.190105 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bab765-b44b-4c04-9efb-4f8a2055aa99 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:31.190375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bab765-b44b-4c04-9efb-4f8a2055aa99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:31.203764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bab765-b44b-4c04-9efb-4f8a2055aa99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:31.212912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-b3bab765-b44b-4c04-9efb-4f8a2055aa99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:31.212912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:31.212912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:31.212912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:31.212912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:31.212912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:31.212912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:31.212912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:31.212912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:31.212912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:31.212912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:31.212912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:31.212912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:31.212912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:31.212912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:31.212912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:31.213937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:31.213937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:31.213937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:31.213937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:31.213937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:31.213937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:31.213937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:31.213937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:31.213937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:31.213937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:31.213937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:31.213937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:31.213937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:31.213937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:31.213937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:31.213937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:31.215356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:31.215356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:31.215356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:31.215356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:31.215356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:31.215356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:31.215356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:31.215356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:31.215356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:31.215356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:31.215356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:31.215356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:31.215356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 200/417] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:31.216021 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acc56f-6617-4f0d-a61e-344a4f3161f0 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:31.216021 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acc56f-6617-4f0d-a61e-344a4f3161f0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:31.216356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acc56f-6617-4f0d-a61e-344a4f3161f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:31.238025 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-55acc56f-6617-4f0d-a61e-344a4f3161f0 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:31.239451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-55acc56f-6617-4f0d-a61e-344a4f3161f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:31.297862 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613e4a4-4429-45f3-a7b9-be7cd55e5c9c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:31.328435 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 201/418] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:31.328435 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acc56f-6617-4f0d-a61e-344a4f3161f0 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:31.355840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a918a-7f33-4cb8-8c98-6cf12e2b766d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:31.356506 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a918a-7f33-4cb8-8c98-6cf12e2b766d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:31.356785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a918a-7f33-4cb8-8c98-6cf12e2b766d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:31.357055 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a918a-7f33-4cb8-8c98-6cf12e2b766d None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:31.358231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a918a-7f33-4cb8-8c98-6cf12e2b766d None admin] RBAC: Authorizing `identity:get_project(project_id=3233ec59a0994511b0032830b4ded6c2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:31.359670 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a918a-7f33-4cb8-8c98-6cf12e2b766d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:31.366046 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 218/419] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:31 2022] GET /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:31.366046 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a918a-7f33-4cb8-8c98-6cf12e2b766d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:31.382517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:31.456145 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:31.462918 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acc56f-6617-4f0d-a61e-344a4f3161f0 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:31.493861 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a918a-7f33-4cb8-8c98-6cf12e2b766d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:31.500257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd2719-2d6a-4ccf-9792-48278a2652a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:31.501059 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd2719-2d6a-4ccf-9792-48278a2652a8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:31.501430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd2719-2d6a-4ccf-9792-48278a2652a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:31.501793 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd2719-2d6a-4ccf-9792-48278a2652a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:31.502637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd2719-2d6a-4ccf-9792-48278a2652a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:31.503054 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd2719-2d6a-4ccf-9792-48278a2652a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:31.504699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd2719-2d6a-4ccf-9792-48278a2652a8 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:31.514336 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 202/420] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:31.515350 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd2719-2d6a-4ccf-9792-48278a2652a8 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:31.540778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e990a-fd38-4868-91d5-096bbc4da12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:31.542730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e990a-fd38-4868-91d5-096bbc4da12c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:31.543011 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e990a-fd38-4868-91d5-096bbc4da12c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:31.543282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e990a-fd38-4868-91d5-096bbc4da12c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:31.556886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e990a-fd38-4868-91d5-096bbc4da12c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:31.565158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-b15e990a-fd38-4868-91d5-096bbc4da12c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:31.565158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:31.565158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:31.565158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:31.565158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:31.565158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:31.565158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:31.565158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:31.565158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:31.565158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:31.565158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:31.565158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:31.565158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:31.565158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:31.565158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:31.565158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:31.566060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:31.566060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:31.566060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:31.566060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:31.566060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:31.566060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:31.566060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:31.566060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:31.566060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:31.566060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:31.566060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:31.566060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:31.566060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:31.566060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:31.566060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:31.566060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:31.567274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:31.567274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:31.567274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:31.567274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:31.567274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:31.567274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:31.567274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:31.567274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:31.567274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:31.567274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:31.567274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:31.567274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:31.567274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 219/421] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:53:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:31.567274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e990a-fd38-4868-91d5-096bbc4da12c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:31.647698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd2719-2d6a-4ccf-9792-48278a2652a8 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:31.648730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:31.676179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e990a-fd38-4868-91d5-096bbc4da12c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:31.696107 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64caec74-3d13-4070-9312-aa61d29525cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:31.697984 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64caec74-3d13-4070-9312-aa61d29525cf None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:31.698409 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64caec74-3d13-4070-9312-aa61d29525cf None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:31.698964 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64caec74-3d13-4070-9312-aa61d29525cf None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2/groups/1ebfe049096c4d839a02d5d9d829c95a/roles/6322fb4a438d4a389c43d05248665e72` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:31.702284 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64caec74-3d13-4070-9312-aa61d29525cf None admin] RBAC: Authorizing `identity:create_grant(project_id=3233ec59a0994511b0032830b4ded6c2, group_id=1ebfe049096c4d839a02d5d9d829c95a, role_id=6322fb4a438d4a389c43d05248665e72)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:31.704775 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64caec74-3d13-4070-9312-aa61d29525cf None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:31.713106 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52570678-c812-43f6-b6a6-5918bb1efde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:31.714037 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52570678-c812-43f6-b6a6-5918bb1efde5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:31.714387 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52570678-c812-43f6-b6a6-5918bb1efde5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:31.714731 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52570678-c812-43f6-b6a6-5918bb1efde5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:31.729106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 203/422] 10.222.0.13 () {58 vars in 1442 bytes} [Sat Jun 25 15:53:31 2022] PUT /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2/groups/1ebfe049096c4d839a02d5d9d829c95a/roles/6322fb4a438d4a389c43d05248665e72 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:53:31.731394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64caec74-3d13-4070-9312-aa61d29525cf None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:31.770436 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:31.849530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52570678-c812-43f6-b6a6-5918bb1efde5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:31.851036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52570678-c812-43f6-b6a6-5918bb1efde5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:31.935147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64caec74-3d13-4070-9312-aa61d29525cf None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:31.982704 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:31.988111 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836ca7e-8323-472f-8cf0-223f8d88efce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:31.989269 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836ca7e-8323-472f-8cf0-223f8d88efce None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:31.989714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836ca7e-8323-472f-8cf0-223f8d88efce None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:31.990121 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836ca7e-8323-472f-8cf0-223f8d88efce None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:31.991318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836ca7e-8323-472f-8cf0-223f8d88efce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:31.992233 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836ca7e-8323-472f-8cf0-223f8d88efce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:31.994227 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836ca7e-8323-472f-8cf0-223f8d88efce None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:32.005745 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 204/423] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:32.011879 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836ca7e-8323-472f-8cf0-223f8d88efce None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:32.023419 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 220/424] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:32.028601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52570678-c812-43f6-b6a6-5918bb1efde5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:32.105177 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:32.126447 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836ca7e-8323-472f-8cf0-223f8d88efce None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:32.163059 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d7bb6-2aed-4fe8-b240-18763dc138c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:32.164094 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d7bb6-2aed-4fe8-b240-18763dc138c6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:32.164612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d7bb6-2aed-4fe8-b240-18763dc138c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:32.164807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52570678-c812-43f6-b6a6-5918bb1efde5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:32.165316 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d7bb6-2aed-4fe8-b240-18763dc138c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:32.211822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11825e4e-a64c-436b-a265-76bc0efbcb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:32.212912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11825e4e-a64c-436b-a265-76bc0efbcb7c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:32.213197 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11825e4e-a64c-436b-a265-76bc0efbcb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:32.213475 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11825e4e-a64c-436b-a265-76bc0efbcb7c None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:32.215380 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11825e4e-a64c-436b-a265-76bc0efbcb7c None admin] RBAC: Authorizing `identity:get_project(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:32.217112 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11825e4e-a64c-436b-a265-76bc0efbcb7c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:32.219889 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 221/425] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:32 2022] GET /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:32.221428 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40dcb8-f02d-43b2-a6ab-1aef58201926 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:32.221712 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40dcb8-f02d-43b2-a6ab-1aef58201926 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:32.221988 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40dcb8-f02d-43b2-a6ab-1aef58201926 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:32.223918 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 222/426] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:32.225426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40dcb8-f02d-43b2-a6ab-1aef58201926 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:32.247260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d7bb6-2aed-4fe8-b240-18763dc138c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:32.249323 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d7bb6-2aed-4fe8-b240-18763dc138c6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:32.291048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:32.346741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:32.364308 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40dcb8-f02d-43b2-a6ab-1aef58201926 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:32.372502 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 205/427] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:32.372502 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae48982-5216-45cc-9ad0-9e071e5a5fe2 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:32.372502 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae48982-5216-45cc-9ad0-9e071e5a5fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:32.372502 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae48982-5216-45cc-9ad0-9e071e5a5fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:32.389313 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-aae48982-5216-45cc-9ad0-9e071e5a5fe2 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:32.390439 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-aae48982-5216-45cc-9ad0-9e071e5a5fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:32.410679 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c81df-e6f3-4e37-b106-6564b9f21d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:32.411717 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c81df-e6f3-4e37-b106-6564b9f21d13 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:32.412003 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c81df-e6f3-4e37-b106-6564b9f21d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:32.412279 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c81df-e6f3-4e37-b106-6564b9f21d13 None admin] PATH_INFO: `/v3/groups/1ebfe049096c4d839a02d5d9d829c95a` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:32.413855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c81df-e6f3-4e37-b106-6564b9f21d13 None admin] RBAC: Authorizing `identity:get_group(group_id=1ebfe049096c4d839a02d5d9d829c95a)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:32.415343 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c81df-e6f3-4e37-b106-6564b9f21d13 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:32.417924 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 223/428] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Jun 25 15:53:32 2022] GET /identity/v3/groups/1ebfe049096c4d839a02d5d9d829c95a => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:32.418650 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c81df-e6f3-4e37-b106-6564b9f21d13 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:32.464734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 206/429] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:32.466258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae48982-5216-45cc-9ad0-9e071e5a5fe2 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:32.557076 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c81df-e6f3-4e37-b106-6564b9f21d13 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:32.570446 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:32.598610 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f055b-f4c3-4ce7-8726-a2abeead388c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:32.599852 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f055b-f4c3-4ce7-8726-a2abeead388c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:32.600326 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f055b-f4c3-4ce7-8726-a2abeead388c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:32.600815 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f055b-f4c3-4ce7-8726-a2abeead388c None admin] PATH_INFO: `/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:32.602233 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f055b-f4c3-4ce7-8726-a2abeead388c None admin] RBAC: Authorizing `identity:get_project(project_id=5743d78b2bc44a2eb50a4bb98ffdf1ea)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:32.603960 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f055b-f4c3-4ce7-8726-a2abeead388c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:32.605965 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 224/430] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:32 2022] GET /identity/v3/projects/5743d78b2bc44a2eb50a4bb98ffdf1ea => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:32.611188 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f055b-f4c3-4ce7-8726-a2abeead388c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:32.625015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae48982-5216-45cc-9ad0-9e071e5a5fe2 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:32.640162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:32.672259 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3609c-0c4b-4868-8f98-ebe940515b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:32.673183 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3609c-0c4b-4868-8f98-ebe940515b02 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:32.673466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3609c-0c4b-4868-8f98-ebe940515b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:32.673740 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3609c-0c4b-4868-8f98-ebe940515b02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:32.674974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3609c-0c4b-4868-8f98-ebe940515b02 None admin] RBAC: Adding query filter params (group.id=1ebfe049096c4d839a02d5d9d829c95a, role.id=8b9bae8f81854757b4f389d3bd22f595, scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:32.675282 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3609c-0c4b-4868-8f98-ebe940515b02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:32.677329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3609c-0c4b-4868-8f98-ebe940515b02 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:32.685727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 207/431] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Jun 25 15:53:32 2022] GET /identity/v3/role_assignments?group.id=1ebfe049096c4d839a02d5d9d829c95a&scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea&role.id=8b9bae8f81854757b4f389d3bd22f595 => generated 611 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:32.687122 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3609c-0c4b-4868-8f98-ebe940515b02 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:32.771290 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f055b-f4c3-4ce7-8726-a2abeead388c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:32.808117 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3609c-0c4b-4868-8f98-ebe940515b02 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:32.848288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590ded8-ce65-4b2d-a23a-b5f4469f7d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:32.849185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590ded8-ce65-4b2d-a23a-b5f4469f7d63 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:32.849750 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590ded8-ce65-4b2d-a23a-b5f4469f7d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:32.850024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590ded8-ce65-4b2d-a23a-b5f4469f7d63 None admin] PATH_INFO: `/v3/groups/1ebfe049096c4d839a02d5d9d829c95a` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:32.851267 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590ded8-ce65-4b2d-a23a-b5f4469f7d63 None admin] RBAC: Authorizing `identity:get_group(group_id=1ebfe049096c4d839a02d5d9d829c95a)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:32.852868 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590ded8-ce65-4b2d-a23a-b5f4469f7d63 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:32.855267 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 208/432] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Jun 25 15:53:32 2022] GET /identity/v3/groups/1ebfe049096c4d839a02d5d9d829c95a => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:32.866789 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590ded8-ce65-4b2d-a23a-b5f4469f7d63 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:32.878780 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5facd-fd10-4d39-b6b4-6dcda19200ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:32.879619 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5facd-fd10-4d39-b6b4-6dcda19200ec None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:32.879905 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5facd-fd10-4d39-b6b4-6dcda19200ec None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:32.880173 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5facd-fd10-4d39-b6b4-6dcda19200ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:32.898519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5facd-fd10-4d39-b6b4-6dcda19200ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:32.907130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-37d5facd-fd10-4d39-b6b4-6dcda19200ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:32.907130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:32.907130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:32.907130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:32.907130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:32.907130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:32.907130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:32.907130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:32.907130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:32.907130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:32.907130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:32.907130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:32.907130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:32.907130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:32.907130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:32.907130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:32.908034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:32.908034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:32.908034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:32.908034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:32.908034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:32.908034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:32.908034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:32.908034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:32.908034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:32.908034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:32.908034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:32.908034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:32.908034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:32.908034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:32.908034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:32.908034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:32.908867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:32.908867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:32.908867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:32.908867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:32.908867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:32.908867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:32.908867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:32.908867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:32.908867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:32.908867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:32.908867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 25 15:53:32.908867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:32.908867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 225/433] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:32.917447 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5facd-fd10-4d39-b6b4-6dcda19200ec None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:32.932259 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:32.970625 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:32.997824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590ded8-ce65-4b2d-a23a-b5f4469f7d63 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:33.044573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd3a9c-4e04-42ea-982f-22385e72863b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:33.045517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5facd-fd10-4d39-b6b4-6dcda19200ec None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:33.045736 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd3a9c-4e04-42ea-982f-22385e72863b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:33.045987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd3a9c-4e04-42ea-982f-22385e72863b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:33.046261 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd3a9c-4e04-42ea-982f-22385e72863b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:33.047100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd3a9c-4e04-42ea-982f-22385e72863b None admin] RBAC: Adding query filter params (group.id=1ebfe049096c4d839a02d5d9d829c95a, role.id=6322fb4a438d4a389c43d05248665e72, scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:33.047412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd3a9c-4e04-42ea-982f-22385e72863b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:33.048902 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd3a9c-4e04-42ea-982f-22385e72863b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:33.056890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 209/434] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Jun 25 15:53:32 2022] GET /identity/v3/role_assignments?group.id=1ebfe049096c4d839a02d5d9d829c95a&scope.project.id=5743d78b2bc44a2eb50a4bb98ffdf1ea&role.id=6322fb4a438d4a389c43d05248665e72 => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:33.057960 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da4600-b5ec-4c15-8518-3f797026c1ca None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:33.058293 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da4600-b5ec-4c15-8518-3f797026c1ca None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:33.058520 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da4600-b5ec-4c15-8518-3f797026c1ca None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:33.059877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 210/435] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:33.073956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c34429-57a8-435d-a0ca-45854905c917 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:33.074267 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c34429-57a8-435d-a0ca-45854905c917 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:33.074514 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c34429-57a8-435d-a0ca-45854905c917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:33.085073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d314b82-8393-4ea0-a492-bda787663659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:33.085283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d314b82-8393-4ea0-a492-bda787663659 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:33.085558 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d314b82-8393-4ea0-a492-bda787663659 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:33.085823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d314b82-8393-4ea0-a492-bda787663659 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:33.086595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d314b82-8393-4ea0-a492-bda787663659 None admin] RBAC: Adding query filter params (name=member) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:33.086903 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d314b82-8393-4ea0-a492-bda787663659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:33.088497 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d314b82-8393-4ea0-a492-bda787663659 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:33.098225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 226/436] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:53:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:33.104074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d314b82-8393-4ea0-a492-bda787663659 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:33.109624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e5c34429-57a8-435d-a0ca-45854905c917 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:33.110845 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e5c34429-57a8-435d-a0ca-45854905c917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:33.196699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 211/437] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:33.224803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c34429-57a8-435d-a0ca-45854905c917 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:33.229643 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d314b82-8393-4ea0-a492-bda787663659 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:33.240810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:33.277921 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:33.281141 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf3475-4b63-4a07-8f5c-054850abd18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:33.281811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf3475-4b63-4a07-8f5c-054850abd18f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:33.282311 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf3475-4b63-4a07-8f5c-054850abd18f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:33.282630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf3475-4b63-4a07-8f5c-054850abd18f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:33.369431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c34429-57a8-435d-a0ca-45854905c917 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:33.370504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf3475-4b63-4a07-8f5c-054850abd18f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:33.372394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf3475-4b63-4a07-8f5c-054850abd18f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:33.412312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677edfa0-ca16-451f-b72c-f5427dcd0fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:33.413569 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677edfa0-ca16-451f-b72c-f5427dcd0fbc None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:33.413897 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677edfa0-ca16-451f-b72c-f5427dcd0fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:33.414187 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677edfa0-ca16-451f-b72c-f5427dcd0fbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:33.431218 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677edfa0-ca16-451f-b72c-f5427dcd0fbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:33.440072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-677edfa0-ca16-451f-b72c-f5427dcd0fbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:53:33.440072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:33.440072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:33.440072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:33.440072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:33.440072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:33.440072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:33.440072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:33.440072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:33.440072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:33.440072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:33.440072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:33.440072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:33.440072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:33.440072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:33.440072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:33.441063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:33.441063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:33.441063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:33.441063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:33.441063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:33.441063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:33.441063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:33.441063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:33.441063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:33.441063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:33.441063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:33.441063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:33.441063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:33.441063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:33.441063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:33.441063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:33.441837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:33.441837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:33.441837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:33.441837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:33.441837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:33.441837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:33.441837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:33.441837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:33.441837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:33.441837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:33.441837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:53:33.441837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:33.441837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 212/438] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:53:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:33.447984 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677edfa0-ca16-451f-b72c-f5427dcd0fbc None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:33.498863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:33.503424 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 227/439] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 25 15:53:33.508814 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf3475-4b63-4a07-8f5c-054850abd18f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:33.574984 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:33.583322 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677edfa0-ca16-451f-b72c-f5427dcd0fbc None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:33.635358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fc573-af0c-42c5-9ddc-7f5d10c5308d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:33.635960 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fc573-af0c-42c5-9ddc-7f5d10c5308d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:33.636244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fc573-af0c-42c5-9ddc-7f5d10c5308d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:33.636568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fc573-af0c-42c5-9ddc-7f5d10c5308d None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:33.637322 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552fc573-af0c-42c5-9ddc-7f5d10c5308d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:33.637628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552fc573-af0c-42c5-9ddc-7f5d10c5308d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:33.639026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552fc573-af0c-42c5-9ddc-7f5d10c5308d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:33.640739 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf3475-4b63-4a07-8f5c-054850abd18f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:33.655721 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 213/440] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:33.656106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fc573-af0c-42c5-9ddc-7f5d10c5308d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:33.689101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bbf20-8519-40e2-8d9b-00eb15abd3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:33.689990 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bbf20-8519-40e2-8d9b-00eb15abd3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:33.690271 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bbf20-8519-40e2-8d9b-00eb15abd3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:33.690537 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bbf20-8519-40e2-8d9b-00eb15abd3c3 None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:33.691752 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bbf20-8519-40e2-8d9b-00eb15abd3c3 None admin] RBAC: Authorizing `identity:get_project(project_id=3233ec59a0994511b0032830b4ded6c2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:33.693249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bbf20-8519-40e2-8d9b-00eb15abd3c3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:33.699719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 228/441] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:33 2022] GET /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:33.701492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bbf20-8519-40e2-8d9b-00eb15abd3c3 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:33.770013 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fc573-af0c-42c5-9ddc-7f5d10c5308d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:33.774500 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:33.806669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec2380-663b-46bd-9a5e-ce8fd5cba85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:33.807720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec2380-663b-46bd-9a5e-ce8fd5cba85c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:33.808142 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec2380-663b-46bd-9a5e-ce8fd5cba85c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:33.808606 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec2380-663b-46bd-9a5e-ce8fd5cba85c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:33.810831 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bbf20-8519-40e2-8d9b-00eb15abd3c3 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:33.842931 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:33.847566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4a098-0a80-4c68-8678-92c9e97afac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:33.848483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4a098-0a80-4c68-8678-92c9e97afac5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:33.848772 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4a098-0a80-4c68-8678-92c9e97afac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:33.849053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4a098-0a80-4c68-8678-92c9e97afac5 None admin] PATH_INFO: `/v3/groups/1ebfe049096c4d839a02d5d9d829c95a` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:33.850248 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4a098-0a80-4c68-8678-92c9e97afac5 None admin] RBAC: Authorizing `identity:get_group(group_id=1ebfe049096c4d839a02d5d9d829c95a)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:33.851738 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4a098-0a80-4c68-8678-92c9e97afac5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:33.853807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 229/442] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Jun 25 15:53:33 2022] GET /identity/v3/groups/1ebfe049096c4d839a02d5d9d829c95a => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:33.859543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4a098-0a80-4c68-8678-92c9e97afac5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:33.896035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec2380-663b-46bd-9a5e-ce8fd5cba85c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:33.897831 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec2380-663b-46bd-9a5e-ce8fd5cba85c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:33.969314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4a098-0a80-4c68-8678-92c9e97afac5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:34.004552 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea689d78-743c-410f-ab4a-04292b3dc2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:34.005205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea689d78-743c-410f-ab4a-04292b3dc2e2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:34.005471 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 214/443] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 25 15:53:34.005561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea689d78-743c-410f-ab4a-04292b3dc2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:34.005757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea689d78-743c-410f-ab4a-04292b3dc2e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:34.006582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea689d78-743c-410f-ab4a-04292b3dc2e2 None admin] RBAC: Adding query filter params (group.id=1ebfe049096c4d839a02d5d9d829c95a, role.id=8b9bae8f81854757b4f389d3bd22f595, scope.project.id=3233ec59a0994511b0032830b4ded6c2) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:34.006900 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea689d78-743c-410f-ab4a-04292b3dc2e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:34.008355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea689d78-743c-410f-ab4a-04292b3dc2e2 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:34.011309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54bdc2-4621-41fb-a4f0-60d53d0a06a0 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:34.011541 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:34.011659 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54bdc2-4621-41fb-a4f0-60d53d0a06a0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:34.011860 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54bdc2-4621-41fb-a4f0-60d53d0a06a0 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:34.013197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 215/444] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:34.013875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54bdc2-4621-41fb-a4f0-60d53d0a06a0 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:34.015700 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 230/445] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Jun 25 15:53:33 2022] GET /identity/v3/role_assignments?group.id=1ebfe049096c4d839a02d5d9d829c95a&scope.project.id=3233ec59a0994511b0032830b4ded6c2&role.id=8b9bae8f81854757b4f389d3bd22f595 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:34.018845 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5299622-4eba-4e05-891a-1634648e50cb None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:34.019131 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5299622-4eba-4e05-891a-1634648e50cb None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:34.019408 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5299622-4eba-4e05-891a-1634648e50cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:34.037789 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c5299622-4eba-4e05-891a-1634648e50cb None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:34.039039 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c5299622-4eba-4e05-891a-1634648e50cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:34.086543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:34.115303 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 231/446] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:34.139993 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5299622-4eba-4e05-891a-1634648e50cb None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:34.148241 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54bdc2-4621-41fb-a4f0-60d53d0a06a0 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:34.191491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813c621-76db-4b97-90b9-f0edf7133d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:34.192365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813c621-76db-4b97-90b9-f0edf7133d0a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:34.192671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813c621-76db-4b97-90b9-f0edf7133d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:34.193448 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813c621-76db-4b97-90b9-f0edf7133d0a None admin] PATH_INFO: `/v3/projects/c8597e0eb76849bdadd0530f7e198ab0` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:34.194417 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7813c621-76db-4b97-90b9-f0edf7133d0a None admin] RBAC: Authorizing `identity:get_project(project_id=c8597e0eb76849bdadd0530f7e198ab0)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:34.195857 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7813c621-76db-4b97-90b9-f0edf7133d0a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:34.198739 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 216/447] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:34 2022] GET /identity/v3/projects/c8597e0eb76849bdadd0530f7e198ab0 => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:34.203497 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813c621-76db-4b97-90b9-f0edf7133d0a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:34.263729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5299622-4eba-4e05-891a-1634648e50cb None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:34.308036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:34.316223 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ac785-a695-4d1b-abe2-b2ff14ea0f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:34.317878 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ac785-a695-4d1b-abe2-b2ff14ea0f46 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:34.317878 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ac785-a695-4d1b-abe2-b2ff14ea0f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:34.318044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813c621-76db-4b97-90b9-f0edf7133d0a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:34.318144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ac785-a695-4d1b-abe2-b2ff14ea0f46 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:34.332115 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ac785-a695-4d1b-abe2-b2ff14ea0f46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:34.340668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-7d5ac785-a695-4d1b-abe2-b2ff14ea0f46 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:34.340668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:34.340668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:34.340668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:34.340668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:34.340668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:34.340668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:34.340668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:34.340668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:34.340668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:34.340668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:34.340668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:34.340668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:34.340668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:34.340668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:34.340668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:34.341560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:34.341560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:34.341560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:34.341560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:34.341560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:34.341560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:34.341560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:34.341560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:34.341560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:34.341560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:34.341560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:34.341560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:34.341560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:34.341560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:34.341560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:34.341560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:34.342524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:34.342524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:34.342524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:34.342524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:34.342524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:34.342524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:34.342524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:34.342524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:34.342524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:34.342524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:34.342524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:34.342524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:34.342524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 232/448] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:53:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:34.348482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ac785-a695-4d1b-abe2-b2ff14ea0f46 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:34.351069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:34.361517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8d14c-26db-4b34-b5c8-a82a0b03c67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:34.362386 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8d14c-26db-4b34-b5c8-a82a0b03c67d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:34.362723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8d14c-26db-4b34-b5c8-a82a0b03c67d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:34.363003 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8d14c-26db-4b34-b5c8-a82a0b03c67d None admin] PATH_INFO: `/v3/groups/a862c3e46e704a2183ff2995a4039670` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:34.364207 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8d14c-26db-4b34-b5c8-a82a0b03c67d None admin] RBAC: Authorizing `identity:get_group(group_id=a862c3e46e704a2183ff2995a4039670)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:34.366760 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8d14c-26db-4b34-b5c8-a82a0b03c67d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:34.369216 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 217/449] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Jun 25 15:53:34 2022] GET /identity/v3/groups/a862c3e46e704a2183ff2995a4039670 => generated 229 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:34.374911 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8d14c-26db-4b34-b5c8-a82a0b03c67d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:34.452990 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ac785-a695-4d1b-abe2-b2ff14ea0f46 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:34.472947 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8d14c-26db-4b34-b5c8-a82a0b03c67d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:34.485835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ad367-afcd-4418-b13a-f02c8df485d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:34.486491 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ad367-afcd-4418-b13a-f02c8df485d2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:34.486778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ad367-afcd-4418-b13a-f02c8df485d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:34.487053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ad367-afcd-4418-b13a-f02c8df485d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:34.487803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ad367-afcd-4418-b13a-f02c8df485d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:34.488109 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ad367-afcd-4418-b13a-f02c8df485d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:34.489533 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ad367-afcd-4418-b13a-f02c8df485d2 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:34.497369 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 233/450] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:34.502813 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ad367-afcd-4418-b13a-f02c8df485d2 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:34.507529 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481af180-8135-4fd2-bf49-f4c212823887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:34.508181 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481af180-8135-4fd2-bf49-f4c212823887 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:34.508450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481af180-8135-4fd2-bf49-f4c212823887 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:34.508728 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481af180-8135-4fd2-bf49-f4c212823887 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:34.509645 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481af180-8135-4fd2-bf49-f4c212823887 None admin] RBAC: Adding query filter params (group.id=a862c3e46e704a2183ff2995a4039670, role.id=b1e13c449fa64b3e88b6832bb767214b, scope.project.id=c8597e0eb76849bdadd0530f7e198ab0) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:34.509954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481af180-8135-4fd2-bf49-f4c212823887 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:34.511482 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481af180-8135-4fd2-bf49-f4c212823887 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:34.518155 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 218/451] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Jun 25 15:53:34 2022] GET /identity/v3/role_assignments?group.id=a862c3e46e704a2183ff2995a4039670&scope.project.id=c8597e0eb76849bdadd0530f7e198ab0&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:34.557565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:34.609064 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ad367-afcd-4418-b13a-f02c8df485d2 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:34.646371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ba8f9-1f55-4291-b2f3-03cd33a40e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:34.647103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ba8f9-1f55-4291-b2f3-03cd33a40e1d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:34.647321 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ba8f9-1f55-4291-b2f3-03cd33a40e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:34.647593 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ba8f9-1f55-4291-b2f3-03cd33a40e1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:34.721101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ba8f9-1f55-4291-b2f3-03cd33a40e1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:34.722520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ba8f9-1f55-4291-b2f3-03cd33a40e1d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:34.773361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:34.824595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 234/452] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 25 15:53:34.832884 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481af180-8135-4fd2-bf49-f4c212823887 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:34.898704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:34.945872 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481af180-8135-4fd2-bf49-f4c212823887 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:34.984433 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6be98-b232-4783-a7e3-1f01432ab0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:34.985243 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6be98-b232-4783-a7e3-1f01432ab0ea None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:34.985525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6be98-b232-4783-a7e3-1f01432ab0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:34.985796 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6be98-b232-4783-a7e3-1f01432ab0ea None admin] PATH_INFO: `/v3/projects/3233ec59a0994511b0032830b4ded6c2` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:34.987045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6be98-b232-4783-a7e3-1f01432ab0ea None admin] RBAC: Authorizing `identity:get_project(project_id=3233ec59a0994511b0032830b4ded6c2)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:34.988534 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6be98-b232-4783-a7e3-1f01432ab0ea None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:34.990219 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 219/453] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:53:34 2022] GET /identity/v3/projects/3233ec59a0994511b0032830b4ded6c2 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:34.995997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ba8f9-1f55-4291-b2f3-03cd33a40e1d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:35.108572 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ba8f9-1f55-4291-b2f3-03cd33a40e1d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:35.144441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c9f7f-842f-4120-882f-63f3d4ecaa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:35.145308 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c9f7f-842f-4120-882f-63f3d4ecaa88 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:35.145588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c9f7f-842f-4120-882f-63f3d4ecaa88 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:35.145864 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c9f7f-842f-4120-882f-63f3d4ecaa88 None admin] PATH_INFO: `/v3/groups/1ebfe049096c4d839a02d5d9d829c95a` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:35.147071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c9f7f-842f-4120-882f-63f3d4ecaa88 None admin] RBAC: Authorizing `identity:get_group(group_id=1ebfe049096c4d839a02d5d9d829c95a)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:35.148530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c9f7f-842f-4120-882f-63f3d4ecaa88 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:35.150284 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 235/454] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Jun 25 15:53:34 2022] GET /identity/v3/groups/1ebfe049096c4d839a02d5d9d829c95a => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:35.156176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6be98-b232-4783-a7e3-1f01432ab0ea None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:35.269133 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6be98-b232-4783-a7e3-1f01432ab0ea None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:35.295902 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:35.305692 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913302bf-c552-432f-98e6-0ca48559140d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:35.306398 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913302bf-c552-432f-98e6-0ca48559140d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:35.306704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913302bf-c552-432f-98e6-0ca48559140d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:35.306967 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913302bf-c552-432f-98e6-0ca48559140d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:35.307799 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913302bf-c552-432f-98e6-0ca48559140d None admin] RBAC: Adding query filter params (group.id=1ebfe049096c4d839a02d5d9d829c95a, role.id=6322fb4a438d4a389c43d05248665e72, scope.project.id=3233ec59a0994511b0032830b4ded6c2) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:35.308104 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913302bf-c552-432f-98e6-0ca48559140d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:35.309582 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913302bf-c552-432f-98e6-0ca48559140d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:35.315515 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 220/455] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Jun 25 15:53:35 2022] GET /identity/v3/role_assignments?group.id=1ebfe049096c4d839a02d5d9d829c95a&scope.project.id=3233ec59a0994511b0032830b4ded6c2&role.id=6322fb4a438d4a389c43d05248665e72 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:37.990533 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22db74-62c4-4392-8fe0-a8e04345cb91 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:37.991075 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22db74-62c4-4392-8fe0-a8e04345cb91 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:37.991075 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22db74-62c4-4392-8fe0-a8e04345cb91 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:37.992780 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 236/456] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:37.998105 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e68144-fd9d-4984-8b9d-b5e2e2fbf974 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:37.998542 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e68144-fd9d-4984-8b9d-b5e2e2fbf974 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:37.998640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e68144-fd9d-4984-8b9d-b5e2e2fbf974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:38.011998 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d3b82-b027-45ed-827c-fe1097b96a5a None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:38.012286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d3b82-b027-45ed-827c-fe1097b96a5a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:38.012598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d3b82-b027-45ed-827c-fe1097b96a5a None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:38.014208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 237/457] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:53:38.021385 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4ad6d-45f9-4985-a061-f63df760c537 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:38.021997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4ad6d-45f9-4985-a061-f63df760c537 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:38.022481 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4ad6d-45f9-4985-a061-f63df760c537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:38.025692 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-72e68144-fd9d-4984-8b9d-b5e2e2fbf974 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:38.026870 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-72e68144-fd9d-4984-8b9d-b5e2e2fbf974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:38.049374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e1b4ad6d-45f9-4985-a061-f63df760c537 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:38.050559 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e1b4ad6d-45f9-4985-a061-f63df760c537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:38.080771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:38.159600 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 221/458] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:53:38.172374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 238/459] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:53:38.188015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e68144-fd9d-4984-8b9d-b5e2e2fbf974 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:38.221345 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4ad6d-45f9-4985-a061-f63df760c537 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:38.319349 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e68144-fd9d-4984-8b9d-b5e2e2fbf974 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:38.333341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4ad6d-45f9-4985-a061-f63df760c537 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:38.341417 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:38.357581 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e788fa4-f166-472e-9ddb-acd2290e9273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:38.358370 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e788fa4-f166-472e-9ddb-acd2290e9273 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:38.358838 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e788fa4-f166-472e-9ddb-acd2290e9273 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:38.359247 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e788fa4-f166-472e-9ddb-acd2290e9273 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:38.370961 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e788fa4-f166-472e-9ddb-acd2290e9273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:38.371194 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a36c7-5c5c-419c-8433-ee49c1929f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:38.371997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a36c7-5c5c-419c-8433-ee49c1929f4d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:38.372279 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a36c7-5c5c-419c-8433-ee49c1929f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:38.372581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a36c7-5c5c-419c-8433-ee49c1929f4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:38.373275 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e788fa4-f166-472e-9ddb-acd2290e9273 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:38.385300 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a36c7-5c5c-419c-8433-ee49c1929f4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:38.386755 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a36c7-5c5c-419c-8433-ee49c1929f4d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:38.397236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-9e788fa4-f166-472e-9ddb-acd2290e9273 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:38.397236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:38.397236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:38.397236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:38.397236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:38.397236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:38.397236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:38.397236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:38.397236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:38.397236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:38.397236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:38.397236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:38.397236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:38.397236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:38.397236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:38.398274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:38.398274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:38.398274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:38.398274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:38.398274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:38.398274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:38.398274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:38.398274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:38.398274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:38.398274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:38.398274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:38.398274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:38.398274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:38.398274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:38.398274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:38.398274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:38.398274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:38.399310 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:38.399310 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:38.399310 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:38.399310 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:38.401254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 222/460] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:38.402664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6481650-f968-4fbd-8697-bd6d868ddb4c None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:38.403086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6481650-f968-4fbd-8697-bd6d868ddb4c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:38.403476 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6481650-f968-4fbd-8697-bd6d868ddb4c None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:38.404908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 223/461] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:38.410277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad836a6-01a7-4dd2-82bf-e61279bf6626 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:38.410396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-273a36c7-5c5c-419c-8433-ee49c1929f4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:38.410396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:38.410396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:38.410396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:38.410396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:38.410396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:38.410396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:38.410396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:38.410396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:38.410396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:38.410396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:38.410396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:38.410396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:38.410396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:38.410396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:38.411172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:38.411172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:38.411172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:38.411172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:38.411172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:38.411172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:38.411172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:38.411172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:38.411172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:38.411172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:38.411172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:38.411172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:38.411172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:38.411172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:38.411172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:38.411172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:38.411172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:38.412037 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:38.412037 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:38.412037 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:38.412037 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:38.413022 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad836a6-01a7-4dd2-82bf-e61279bf6626 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:38.413493 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad836a6-01a7-4dd2-82bf-e61279bf6626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:38.416205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 239/462] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:38.417341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a36c7-5c5c-419c-8433-ee49c1929f4d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:38.420331 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:38.436087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2ad836a6-01a7-4dd2-82bf-e61279bf6626 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:38.437663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2ad836a6-01a7-4dd2-82bf-e61279bf6626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:38.527990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 224/463] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:53:38.527990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad836a6-01a7-4dd2-82bf-e61279bf6626 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:38.563622 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a36c7-5c5c-419c-8433-ee49c1929f4d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:38.621260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f9995-0166-4ad8-bebc-17c8b2be59dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:38.623107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f9995-0166-4ad8-bebc-17c8b2be59dc None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:38.623796 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f9995-0166-4ad8-bebc-17c8b2be59dc None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:38.625053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f9995-0166-4ad8-bebc-17c8b2be59dc None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:38.626713 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f9995-0166-4ad8-bebc-17c8b2be59dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:38.627736 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f9995-0166-4ad8-bebc-17c8b2be59dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:38.630406 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f9995-0166-4ad8-bebc-17c8b2be59dc None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:38.646493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 240/464] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:38.647727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f9995-0166-4ad8-bebc-17c8b2be59dc None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:38.653687 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad836a6-01a7-4dd2-82bf-e61279bf6626 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:38.655389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:38.690976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4eab3-0a40-4bf1-b295-41abab6a1f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:38.691649 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4eab3-0a40-4bf1-b295-41abab6a1f7f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:38.691933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4eab3-0a40-4bf1-b295-41abab6a1f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:38.692210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4eab3-0a40-4bf1-b295-41abab6a1f7f None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:38.692990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b4eab3-0a40-4bf1-b295-41abab6a1f7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:38.693298 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b4eab3-0a40-4bf1-b295-41abab6a1f7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:38.694696 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b4eab3-0a40-4bf1-b295-41abab6a1f7f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:38.711623 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 225/465] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:38.712839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4eab3-0a40-4bf1-b295-41abab6a1f7f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:38.717307 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:38.762109 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f9995-0166-4ad8-bebc-17c8b2be59dc None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:38.800963 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d7dbe-00c0-47cb-a2aa-f793d6f75ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:38.801753 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d7dbe-00c0-47cb-a2aa-f793d6f75ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:38.802035 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d7dbe-00c0-47cb-a2aa-f793d6f75ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:38.802307 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d7dbe-00c0-47cb-a2aa-f793d6f75ff6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:38.812930 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d7dbe-00c0-47cb-a2aa-f793d6f75ff6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:38.814385 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d7dbe-00c0-47cb-a2aa-f793d6f75ff6 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:38.816197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4eab3-0a40-4bf1-b295-41abab6a1f7f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:38.832795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-297d7dbe-00c0-47cb-a2aa-f793d6f75ff6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:38.832795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:38.832795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:38.832795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:38.832795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:38.832795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:38.832795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:38.832795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:38.832795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:38.832795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:38.832795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:38.832795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:38.832795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:38.832795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:38.832795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:38.833696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:38.833696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:38.833696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:38.833696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:38.833696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:38.833696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:38.833696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:38.833696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:38.833696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:38.833696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:38.833696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:38.833696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:38.833696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:38.833696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:38.833696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:38.833696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:38.833696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:38.834547 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:38.834547 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:38.834547 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:38.834547 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:38.834547 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 241/466] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:38.834803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d7dbe-00c0-47cb-a2aa-f793d6f75ff6 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:38.856987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54075cb8-8e43-49cc-aa5a-56d15e501bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:38.858019 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54075cb8-8e43-49cc-aa5a-56d15e501bfe None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:38.858479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54075cb8-8e43-49cc-aa5a-56d15e501bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:38.858925 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54075cb8-8e43-49cc-aa5a-56d15e501bfe None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:38.859808 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54075cb8-8e43-49cc-aa5a-56d15e501bfe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:38.861543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54075cb8-8e43-49cc-aa5a-56d15e501bfe None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:38.884101 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 226/467] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Jun 25 15:53:38 2022] POST /identity/v3/users => generated 246 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:53:38.886448 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79123dc-a0bd-4ff8-bdef-ce26634a7617 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:38.887116 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79123dc-a0bd-4ff8-bdef-ce26634a7617 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:38.887748 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79123dc-a0bd-4ff8-bdef-ce26634a7617 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:38.890549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 227/468] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:38.892339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79123dc-a0bd-4ff8-bdef-ce26634a7617 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:38.928336 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:38.995288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d7dbe-00c0-47cb-a2aa-f793d6f75ff6 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:39.018385 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:39.041281 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79123dc-a0bd-4ff8-bdef-ce26634a7617 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:39.049526 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4cebb6-3ba8-4a58-a559-e3178311154a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:39.050518 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cebb6-3ba8-4a58-a559-e3178311154a None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:39.050972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cebb6-3ba8-4a58-a559-e3178311154a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:39.051435 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4cebb6-3ba8-4a58-a559-e3178311154a None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:39.052323 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4cebb6-3ba8-4a58-a559-e3178311154a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:39.054079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4cebb6-3ba8-4a58-a559-e3178311154a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:39.079968 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 242/469] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Jun 25 15:53:38 2022] POST /identity/v3/users => generated 248 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:53:39.081566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb08d43-f74b-4eca-8bb9-c288f6c95bcf None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:39.082026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb08d43-f74b-4eca-8bb9-c288f6c95bcf None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:39.082492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb08d43-f74b-4eca-8bb9-c288f6c95bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:39.090939 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c7a6a-acd3-4a0a-82a7-7f1e830b54ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:39.091677 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c7a6a-acd3-4a0a-82a7-7f1e830b54ab None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:39.091963 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c7a6a-acd3-4a0a-82a7-7f1e830b54ab None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:39.092288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c7a6a-acd3-4a0a-82a7-7f1e830b54ab None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:39.093056 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c7a6a-acd3-4a0a-82a7-7f1e830b54ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:39.093411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c7a6a-acd3-4a0a-82a7-7f1e830b54ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:39.094880 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c7a6a-acd3-4a0a-82a7-7f1e830b54ab None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:39.100542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7cb08d43-f74b-4eca-8bb9-c288f6c95bcf None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:39.101774 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7cb08d43-f74b-4eca-8bb9-c288f6c95bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:39.109741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 228/470] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:39.115898 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c7a6a-acd3-4a0a-82a7-7f1e830b54ab None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:39.191578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 243/471] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:39.220022 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb08d43-f74b-4eca-8bb9-c288f6c95bcf None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:39.226620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:39.251765 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c7a6a-acd3-4a0a-82a7-7f1e830b54ab None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:39.294198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bfd80d-1990-4084-aa31-a7f40b95650d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:39.295364 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfd80d-1990-4084-aa31-a7f40b95650d None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:39.295952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfd80d-1990-4084-aa31-a7f40b95650d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:39.296450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfd80d-1990-4084-aa31-a7f40b95650d None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:39.297416 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bfd80d-1990-4084-aa31-a7f40b95650d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:39.299093 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bfd80d-1990-4084-aa31-a7f40b95650d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:39.323226 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 229/472] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Jun 25 15:53:39 2022] POST /identity/v3/users => generated 248 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:53:39.345771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:39.350386 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb08d43-f74b-4eca-8bb9-c288f6c95bcf None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:39.391075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590f110-76cd-4d79-b1bc-bdb23a74a8b4 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:39.391999 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590f110-76cd-4d79-b1bc-bdb23a74a8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:39.392577 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590f110-76cd-4d79-b1bc-bdb23a74a8b4 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:39.394484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 230/473] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:39.401371 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc8f0b-7793-452d-882d-22e1b9ec64bc None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:39.401688 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc8f0b-7793-452d-882d-22e1b9ec64bc None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:39.401920 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc8f0b-7793-452d-882d-22e1b9ec64bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:39.402750 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c60908-2620-461c-bdc7-12afae210cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:39.403845 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c60908-2620-461c-bdc7-12afae210cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:39.404346 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c60908-2620-461c-bdc7-12afae210cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:39.404839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c60908-2620-461c-bdc7-12afae210cf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:39.420609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c60908-2620-461c-bdc7-12afae210cf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:39.422599 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c60908-2620-461c-bdc7-12afae210cf8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:39.424085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-43fc8f0b-7793-452d-882d-22e1b9ec64bc None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:39.425347 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-43fc8f0b-7793-452d-882d-22e1b9ec64bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:39.446763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-42c60908-2620-461c-bdc7-12afae210cf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:39.446763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:39.446763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:39.446763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:39.446763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:39.446763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:39.446763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:39.446763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:39.446763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:39.446763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:39.446763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:39.446763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:39.446763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:39.446763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:39.446763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:39.447874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:39.447874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:39.447874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:39.447874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:39.447874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:39.447874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:39.447874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:39.447874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:39.447874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:39.447874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:39.447874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:39.447874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:39.447874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:39.447874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:39.447874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:39.447874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:39.447874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:39.448941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:39.448941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:39.448941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:39.448941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:39.456696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 244/474] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:39.458906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c60908-2620-461c-bdc7-12afae210cf8 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:39.529321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 231/475] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:53:39.560321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc8f0b-7793-452d-882d-22e1b9ec64bc None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:39.591382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c60908-2620-461c-bdc7-12afae210cf8 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:39.643627 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:39.644615 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbd0d6-c110-4f2c-a2c5-b0f43031f4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:39.645226 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbd0d6-c110-4f2c-a2c5-b0f43031f4fe None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:39.645544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbd0d6-c110-4f2c-a2c5-b0f43031f4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:39.645794 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbd0d6-c110-4f2c-a2c5-b0f43031f4fe None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:39.647411 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfbd0d6-c110-4f2c-a2c5-b0f43031f4fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:39.647411 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfbd0d6-c110-4f2c-a2c5-b0f43031f4fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:39.648208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfbd0d6-c110-4f2c-a2c5-b0f43031f4fe None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:39.669003 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 245/476] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:39.673233 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbd0d6-c110-4f2c-a2c5-b0f43031f4fe None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:39.694377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:39.711475 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc8f0b-7793-452d-882d-22e1b9ec64bc None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:39.761181 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676734f-5342-45f4-9f36-3c17826f5958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:39.761921 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676734f-5342-45f4-9f36-3c17826f5958 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:39.762204 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676734f-5342-45f4-9f36-3c17826f5958 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:39.762525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676734f-5342-45f4-9f36-3c17826f5958 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:39.763286 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1676734f-5342-45f4-9f36-3c17826f5958 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:39.764765 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1676734f-5342-45f4-9f36-3c17826f5958 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:39.785136 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.sql.core [None req-1676734f-5342-45f4-9f36-3c17826f5958 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 25 15:53:39.785136 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 25 15:53:39.785136 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [parameters: {'id': '8f0ab9285c4345eeac13c81c042a4170', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 25 15:53:39.785136 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80125) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 25 15:53:39.796636 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbd0d6-c110-4f2c-a2c5-b0f43031f4fe None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:39.810412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-1676734f-5342-45f4-9f36-3c17826f5958 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 25 15:53:39.810412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:39.810412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 25 15:53:39.810412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 25 15:53:39.810412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 25 15:53:39.810412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 25 15:53:39.810412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 25 15:53:39.810412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application result = self._query(query) Jun 25 15:53:39.810412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 25 15:53:39.810412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application conn.query(q) Jun 25 15:53:39.810412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 25 15:53:39.810412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 25 15:53:39.810412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 25 15:53:39.810412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application result.read() Jun 25 15:53:39.810412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 25 15:53:39.811358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 25 15:53:39.811358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 25 15:53:39.811358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 25 15:53:39.811358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 25 15:53:39.811358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 25 15:53:39.811358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 25 15:53:39.811358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 25 15:53:39.811358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 25 15:53:39.811358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:39.811358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 25 15:53:39.811358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:39.811358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:39.811358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 25 15:53:39.811358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 25 15:53:39.811358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 25 15:53:39.811358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return ref.to_dict() Jun 25 15:53:39.811358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 25 15:53:39.811358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application next(self.gen) Jun 25 15:53:39.813029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 25 15:53:39.813029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application yield resource Jun 25 15:53:39.813029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 25 15:53:39.813029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application next(self.gen) Jun 25 15:53:39.813029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 25 15:53:39.813029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self.session.rollback() Jun 25 15:53:39.813029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 25 15:53:39.813029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self.force_reraise() Jun 25 15:53:39.813029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 25 15:53:39.813029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise self.value Jun 25 15:53:39.813029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 25 15:53:39.813029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 25 15:53:39.813029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 25 15:53:39.813029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application session.commit() Jun 25 15:53:39.813029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 25 15:53:39.813029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 25 15:53:39.813029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 25 15:53:39.813940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._prepare_impl() Jun 25 15:53:39.813940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 25 15:53:39.813940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self.session.flush() Jun 25 15:53:39.813940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 25 15:53:39.813940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._flush(objects) Jun 25 15:53:39.813940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 25 15:53:39.813940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 25 15:53:39.813940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 25 15:53:39.813940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application compat.raise_( Jun 25 15:53:39.813940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 25 15:53:39.813940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception Jun 25 15:53:39.813940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 25 15:53:39.813940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application flush_context.execute() Jun 25 15:53:39.813940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 25 15:53:39.813940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rec.execute(self) Jun 25 15:53:39.813940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 25 15:53:39.813940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 25 15:53:39.814784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 25 15:53:39.814784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 25 15:53:39.814784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 25 15:53:39.814784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 25 15:53:39.814784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 25 15:53:39.814784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 25 15:53:39.814784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 25 15:53:39.814784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 25 15:53:39.814784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 25 15:53:39.814784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 25 15:53:39.814784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 25 15:53:39.814784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 25 15:53:39.814784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 25 15:53:39.814784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 25 15:53:39.814784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 25 15:53:39.814784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception Jun 25 15:53:39.815587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 25 15:53:39.815587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 25 15:53:39.815587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 25 15:53:39.815587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 25 15:53:39.815587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 25 15:53:39.815587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application result = self._query(query) Jun 25 15:53:39.815587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 25 15:53:39.815587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application conn.query(q) Jun 25 15:53:39.815587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 25 15:53:39.815587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 25 15:53:39.815587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 25 15:53:39.815587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application result.read() Jun 25 15:53:39.815587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 25 15:53:39.815587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 25 15:53:39.815587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 25 15:53:39.815587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 25 15:53:39.817453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 25 15:53:39.817453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 25 15:53:39.817453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 25 15:53:39.817453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 25 15:53:39.817453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 25 15:53:39.817453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 25 15:53:39.817453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [parameters: {'id': '8f0ab9285c4345eeac13c81c042a4170', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 25 15:53:39.817453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 25 15:53:39.817453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:39.817453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:53:39.817453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:39.817453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:39.817453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:39.817453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:39.817453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:39.817453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:39.818991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:39.818991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:39.818991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:39.818991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:39.818991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:39.818991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:39.818991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 25 15:53:39.818991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 25 15:53:39.818991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:39.818991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:39.818991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 25 15:53:39.818991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 25 15:53:39.818991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 25 15:53:39.818991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 25 15:53:39.818991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 25 15:53:39.818991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:39.820015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 232/477] 10.222.0.13 () {60 vars in 1131 bytes} [Sat Jun 25 15:53:39 2022] POST /identity/v3/roles => generated 144 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:39.820015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676734f-5342-45f4-9f36-3c17826f5958 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:39.865062 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda2b2d-adfd-4011-a195-d5152edcb8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:39.865883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda2b2d-adfd-4011-a195-d5152edcb8b7 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:39.866287 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda2b2d-adfd-4011-a195-d5152edcb8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:39.866655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda2b2d-adfd-4011-a195-d5152edcb8b7 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:39.867559 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda2b2d-adfd-4011-a195-d5152edcb8b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:39.869471 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda2b2d-adfd-4011-a195-d5152edcb8b7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:39.900366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 246/478] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Jun 25 15:53:39 2022] POST /identity/v3/users => generated 249 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 15:53:39.998432 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676734f-5342-45f4-9f36-3c17826f5958 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:40.028644 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:40.040159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a74efe-964c-4930-a2d5-3ce8b4eb163b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:40.041225 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a74efe-964c-4930-a2d5-3ce8b4eb163b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:40.041507 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a74efe-964c-4930-a2d5-3ce8b4eb163b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:40.042031 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a74efe-964c-4930-a2d5-3ce8b4eb163b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:40.053953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a74efe-964c-4930-a2d5-3ce8b4eb163b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:40.062893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-86a74efe-964c-4930-a2d5-3ce8b4eb163b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:40.062893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:40.062893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:40.062893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:40.062893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:40.062893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:40.062893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:40.062893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:40.062893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:40.062893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:40.062893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:40.062893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:40.062893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:40.062893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:40.062893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:40.062893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:40.063858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:40.063858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:40.063858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:40.063858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:40.063858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:40.063858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:40.063858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:40.063858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:40.063858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:40.063858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:40.063858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:40.063858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:40.063858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:40.063858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:40.063858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:40.063858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:40.065005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:40.065005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:40.065005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:40.065005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:40.065005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:40.065005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:40.065005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:40.065005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:40.065005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:40.065005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:40.065005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 25 15:53:40.065005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:40.065005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 233/479] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:53:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:40.073574 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda2b2d-adfd-4011-a195-d5152edcb8b7 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:40.151786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:40.187985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda2b2d-adfd-4011-a195-d5152edcb8b7 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:40.222652 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b1337-d354-4e3b-af31-0acef2f327d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:40.223321 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b1337-d354-4e3b-af31-0acef2f327d2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:40.223667 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b1337-d354-4e3b-af31-0acef2f327d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:40.223943 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b1337-d354-4e3b-af31-0acef2f327d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:40.224726 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b1337-d354-4e3b-af31-0acef2f327d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:40.225030 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b1337-d354-4e3b-af31-0acef2f327d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:40.226468 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b1337-d354-4e3b-af31-0acef2f327d2 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:40.235014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 247/480] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:41.709649 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04593e8-491a-4d06-b427-7b731185d8d3 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:41.710175 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04593e8-491a-4d06-b427-7b731185d8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:41.710175 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04593e8-491a-4d06-b427-7b731185d8d3 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:41.711846 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 234/481] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:41.717963 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06dfecd-3883-48f2-adbf-f105d4238c0a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:41.718789 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06dfecd-3883-48f2-adbf-f105d4238c0a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:41.719261 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06dfecd-3883-48f2-adbf-f105d4238c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:41.745388 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a06dfecd-3883-48f2-adbf-f105d4238c0a None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:41.746925 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a06dfecd-3883-48f2-adbf-f105d4238c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:41.773423 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17a035-00ce-4816-9856-08a3a660d612 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:41.773768 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17a035-00ce-4816-9856-08a3a660d612 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:41.774055 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17a035-00ce-4816-9856-08a3a660d612 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:41.776023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 235/482] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:53:41.781686 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921eb7d4-e5cc-4bf3-ae2f-678f6b739ae3 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:41.781980 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921eb7d4-e5cc-4bf3-ae2f-678f6b739ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:41.782258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921eb7d4-e5cc-4bf3-ae2f-678f6b739ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:41.805248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-921eb7d4-e5cc-4bf3-ae2f-678f6b739ae3 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:41.806461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-921eb7d4-e5cc-4bf3-ae2f-678f6b739ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:41.842274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 248/483] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:41.890385 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 236/484] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:53:41.897561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06dfecd-3883-48f2-adbf-f105d4238c0a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:41.964763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:41.973012 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921eb7d4-e5cc-4bf3-ae2f-678f6b739ae3 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:42.045103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06dfecd-3883-48f2-adbf-f105d4238c0a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:42.090970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921eb7d4-e5cc-4bf3-ae2f-678f6b739ae3 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:42.092077 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03508cff-0d02-4016-8f59-69e1697de5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:42.093075 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03508cff-0d02-4016-8f59-69e1697de5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:42.094147 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03508cff-0d02-4016-8f59-69e1697de5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:42.094446 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03508cff-0d02-4016-8f59-69e1697de5e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:42.107487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03508cff-0d02-4016-8f59-69e1697de5e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:42.111005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:42.117423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-03508cff-0d02-4016-8f59-69e1697de5e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:42.117423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:42.117423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:42.117423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:42.117423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:42.117423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:42.117423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:42.117423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:42.117423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:42.117423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:42.117423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:42.117423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:42.117423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:42.117423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:42.117423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:42.117423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:42.118483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:42.118483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:42.118483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:42.118483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:42.118483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:42.118483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:42.118483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:42.118483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:42.118483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:42.118483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:42.118483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:42.118483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:42.118483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:42.118483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:42.118483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:42.118483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:42.119274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:42.119274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:42.119274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:42.119274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:42.119274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:42.119274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:42.119274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:42.119274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:42.119274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:42.119274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:42.119274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:42.119274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:42.120867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 249/485] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:53:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:42.121805 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66166c0b-6480-4c5e-ad2e-1e9a56d724c4 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:42.122088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66166c0b-6480-4c5e-ad2e-1e9a56d724c4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:42.122365 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66166c0b-6480-4c5e-ad2e-1e9a56d724c4 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:42.123639 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 250/486] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:42.127163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66166c0b-6480-4c5e-ad2e-1e9a56d724c4 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:42.140506 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bfe51b-0b4e-4c20-8210-26f2c3d0c7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:42.141793 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfe51b-0b4e-4c20-8210-26f2c3d0c7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:42.142145 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfe51b-0b4e-4c20-8210-26f2c3d0c7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:42.142418 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bfe51b-0b4e-4c20-8210-26f2c3d0c7b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:42.156995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bfe51b-0b4e-4c20-8210-26f2c3d0c7b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:42.167301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-44bfe51b-0b4e-4c20-8210-26f2c3d0c7b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:42.167301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:42.167301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:42.167301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:42.167301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:42.167301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:42.167301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:42.167301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:42.167301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:42.167301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:42.167301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:42.167301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:42.167301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:42.167301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:42.167301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:42.167301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:42.169420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:42.169420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:42.169420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:42.169420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:42.169420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:42.169420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:42.169420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:42.169420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:42.169420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:42.169420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:42.169420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:42.169420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:42.169420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:42.169420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:42.169420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:42.169420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:42.171318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:42.171318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:42.171318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:42.171318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:42.171318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:42.171318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:42.171318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:42.171318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:42.171318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:42.171318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:42.171318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:42.171318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:42.171318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 237/487] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:53:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:42.172885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa4132-e004-4463-98e1-6dbd73dde299 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:42.172885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa4132-e004-4463-98e1-6dbd73dde299 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:42.172885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa4132-e004-4463-98e1-6dbd73dde299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:42.193998 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f3aa4132-e004-4463-98e1-6dbd73dde299 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:42.195245 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f3aa4132-e004-4463-98e1-6dbd73dde299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:42.257618 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66166c0b-6480-4c5e-ad2e-1e9a56d724c4 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:42.282404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:42.290859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 238/488] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:42.291906 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa4132-e004-4463-98e1-6dbd73dde299 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:42.320514 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c9f14-d756-4455-aa23-880cd9792487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:42.321286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c9f14-d756-4455-aa23-880cd9792487 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:42.322434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c9f14-d756-4455-aa23-880cd9792487 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:42.322549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c9f14-d756-4455-aa23-880cd9792487 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:42.323333 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c9f14-d756-4455-aa23-880cd9792487 None admin] RBAC: Adding query filter params (name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:42.323687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c9f14-d756-4455-aa23-880cd9792487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:42.325155 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c9f14-d756-4455-aa23-880cd9792487 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:42.342345 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 251/489] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:53:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:42.343275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c9f14-d756-4455-aa23-880cd9792487 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:42.414369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa4132-e004-4463-98e1-6dbd73dde299 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:42.427228 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:42.456878 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c9f14-d756-4455-aa23-880cd9792487 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:42.463487 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71437ba-d075-45e5-a62f-2de392be3b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:42.464369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71437ba-d075-45e5-a62f-2de392be3b3b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:42.464809 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71437ba-d075-45e5-a62f-2de392be3b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:42.465747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71437ba-d075-45e5-a62f-2de392be3b3b None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:42.466653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71437ba-d075-45e5-a62f-2de392be3b3b None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:42.467112 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71437ba-d075-45e5-a62f-2de392be3b3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:42.469429 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71437ba-d075-45e5-a62f-2de392be3b3b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:42.480995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 239/490] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:53:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:42.482355 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71437ba-d075-45e5-a62f-2de392be3b3b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:42.496673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796c456-ffe8-42a3-8527-324f60368a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:42.497483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796c456-ffe8-42a3-8527-324f60368a57 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:42.497828 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796c456-ffe8-42a3-8527-324f60368a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:42.498111 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796c456-ffe8-42a3-8527-324f60368a57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:42.510075 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796c456-ffe8-42a3-8527-324f60368a57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:42.518784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-3796c456-ffe8-42a3-8527-324f60368a57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:42.518784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:42.518784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:42.518784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:42.518784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:42.518784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:42.518784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:42.518784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:42.518784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:42.518784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:42.518784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:42.518784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:42.518784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:42.518784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:42.518784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:42.518784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:42.519855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:42.519855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:42.519855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:42.519855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:42.519855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:42.519855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:42.519855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:42.519855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:42.519855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:42.519855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:42.519855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:42.519855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:42.519855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:42.519855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:42.519855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:42.519855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:42.520788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:42.520788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:42.520788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:42.520788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:42.520788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:42.520788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:42.520788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:42.520788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:42.520788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:42.520788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:42.520788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:42.520788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:42.520788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 252/491] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:53:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:42.527659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796c456-ffe8-42a3-8527-324f60368a57 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:42.580447 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:42.602312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71437ba-d075-45e5-a62f-2de392be3b3b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:42.638885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa2d84-1389-47a9-a05f-c724470a9089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:42.639691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa2d84-1389-47a9-a05f-c724470a9089 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:42.640001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa2d84-1389-47a9-a05f-c724470a9089 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:42.640245 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa2d84-1389-47a9-a05f-c724470a9089 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:42.645538 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796c456-ffe8-42a3-8527-324f60368a57 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:42.651910 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa2d84-1389-47a9-a05f-c724470a9089 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:42.653714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa2d84-1389-47a9-a05f-c724470a9089 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:42.671460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-34aa2d84-1389-47a9-a05f-c724470a9089 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:42.671460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:42.671460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:42.671460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:42.671460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:42.671460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:42.671460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:42.671460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:42.671460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:42.671460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:42.671460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:42.671460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:42.671460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:42.671460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:42.671460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:42.672335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:42.672335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:42.672335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:42.672335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:42.672335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:42.672335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:42.672335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:42.672335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:42.672335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:42.672335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:42.672335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:42.672335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:42.672335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:42.672335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:42.672335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:42.672335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:42.672335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:42.673339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:42.673339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:42.673339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:42.673339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:42.675456 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 240/492] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:42.676362 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa2d84-1389-47a9-a05f-c724470a9089 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:42.685714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cada6e-4f18-4c7a-b075-9a0a679abc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:42.686512 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cada6e-4f18-4c7a-b075-9a0a679abc03 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:42.686797 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cada6e-4f18-4c7a-b075-9a0a679abc03 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:42.687072 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cada6e-4f18-4c7a-b075-9a0a679abc03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:42.699172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cada6e-4f18-4c7a-b075-9a0a679abc03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:42.700674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cada6e-4f18-4c7a-b075-9a0a679abc03 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:42.706927 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:42.717735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-e1cada6e-4f18-4c7a-b075-9a0a679abc03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:42.717735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:42.717735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:42.717735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:42.717735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:42.717735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:42.717735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:42.717735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:42.717735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:42.717735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:42.717735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:42.717735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:42.717735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:42.717735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:42.717735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:42.719140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:42.719140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:42.719140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:42.719140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:42.719140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:42.719140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:42.719140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:42.719140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:42.719140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:42.719140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:42.719140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:42.719140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:42.719140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:42.719140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:42.719140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:42.719140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:42.719140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:42.720052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:42.720052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:42.720052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:42.720052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:42.720052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 253/493] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:42.720471 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a029c1-b9c7-4bfe-8389-07445fb9636a None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:42.720726 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a029c1-b9c7-4bfe-8389-07445fb9636a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:42.721005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a029c1-b9c7-4bfe-8389-07445fb9636a None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:42.722390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 254/494] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:42.722934 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a029c1-b9c7-4bfe-8389-07445fb9636a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:42.782190 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa2d84-1389-47a9-a05f-c724470a9089 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:42.815596 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022809a5-d1fc-410c-920f-48f0ef4965cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:42.816265 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022809a5-d1fc-410c-920f-48f0ef4965cb None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:42.816582 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022809a5-d1fc-410c-920f-48f0ef4965cb None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:42.816859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022809a5-d1fc-410c-920f-48f0ef4965cb None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:42.817611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022809a5-d1fc-410c-920f-48f0ef4965cb None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:42.817915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022809a5-d1fc-410c-920f-48f0ef4965cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:42.819299 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022809a5-d1fc-410c-920f-48f0ef4965cb None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:42.820400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:42.827702 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 241/495] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:53:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:42.827839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a029c1-b9c7-4bfe-8389-07445fb9636a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:42.829035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e00cdf-130d-43d8-886f-43f8504c2a88 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:42.829322 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e00cdf-130d-43d8-886f-43f8504c2a88 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:42.829598 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e00cdf-130d-43d8-886f-43f8504c2a88 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:42.832535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 242/496] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:42.832535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e00cdf-130d-43d8-886f-43f8504c2a88 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:42.864089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2455fb-e98a-459d-8c41-f62c99084ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:42.864787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2455fb-e98a-459d-8c41-f62c99084ade None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:42.865070 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2455fb-e98a-459d-8c41-f62c99084ade None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:42.865346 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2455fb-e98a-459d-8c41-f62c99084ade None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:42.866071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2455fb-e98a-459d-8c41-f62c99084ade None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:42.866383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2455fb-e98a-459d-8c41-f62c99084ade None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:42.867760 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2455fb-e98a-459d-8c41-f62c99084ade None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:42.879612 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 255/497] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:42.880841 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4529125-1988-4b17-bf70-34f70c83713b None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:42.881129 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4529125-1988-4b17-bf70-34f70c83713b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:42.881403 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4529125-1988-4b17-bf70-34f70c83713b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:42.897261 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e4529125-1988-4b17-bf70-34f70c83713b None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:42.898445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e4529125-1988-4b17-bf70-34f70c83713b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:42.918452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:42.935254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e00cdf-130d-43d8-886f-43f8504c2a88 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:42.970429 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 256/498] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:42.971263 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4529125-1988-4b17-bf70-34f70c83713b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:42.974978 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237de0c4-19e1-4f3d-8d79-578bae2c6db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:42.975641 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237de0c4-19e1-4f3d-8d79-578bae2c6db4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:42.975918 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237de0c4-19e1-4f3d-8d79-578bae2c6db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:42.976186 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237de0c4-19e1-4f3d-8d79-578bae2c6db4 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:42.976934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237de0c4-19e1-4f3d-8d79-578bae2c6db4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:42.977235 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237de0c4-19e1-4f3d-8d79-578bae2c6db4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:42.978933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237de0c4-19e1-4f3d-8d79-578bae2c6db4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:42.990944 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 243/499] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:42.992829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdc184-97fd-4b16-b6e2-c89e8e8b304d None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:42.993383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdc184-97fd-4b16-b6e2-c89e8e8b304d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:42.993917 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdc184-97fd-4b16-b6e2-c89e8e8b304d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:43.010479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-55cdc184-97fd-4b16-b6e2-c89e8e8b304d None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:43.011691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-55cdc184-97fd-4b16-b6e2-c89e8e8b304d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:43.058004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:43.076651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4529125-1988-4b17-bf70-34f70c83713b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:43.080105 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 244/500] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:43.080952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdc184-97fd-4b16-b6e2-c89e8e8b304d None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:43.113090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8702e-627c-4a35-a176-0befd9c2f636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:43.113893 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8702e-627c-4a35-a176-0befd9c2f636 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:43.114200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8702e-627c-4a35-a176-0befd9c2f636 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:43.114480 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8702e-627c-4a35-a176-0befd9c2f636 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:43.124596 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8702e-627c-4a35-a176-0befd9c2f636 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:43.126054 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8702e-627c-4a35-a176-0befd9c2f636 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:43.144364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-41e8702e-627c-4a35-a176-0befd9c2f636 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:43.144364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:43.144364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:43.144364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:43.144364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:43.144364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:43.144364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:43.144364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:43.144364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:43.144364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:43.144364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:43.144364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:43.144364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:43.144364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:43.144364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:43.145236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:43.145236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:43.145236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:43.145236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:43.145236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:43.145236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:43.145236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:43.145236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:43.145236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:43.145236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:43.145236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:43.145236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:43.145236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:43.145236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:43.145236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:43.145236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:43.145236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:43.146064 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:43.146064 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:43.146064 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:43.146064 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:43.146064 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 257/501] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:43.146325 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8702e-627c-4a35-a176-0befd9c2f636 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:43.159753 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:43.186183 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdc184-97fd-4b16-b6e2-c89e8e8b304d None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:43.222306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013bd0b-922f-4cd9-8b12-69a93dfd65d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:43.223637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013bd0b-922f-4cd9-8b12-69a93dfd65d9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:43.224141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013bd0b-922f-4cd9-8b12-69a93dfd65d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:43.224683 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013bd0b-922f-4cd9-8b12-69a93dfd65d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:43.234676 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013bd0b-922f-4cd9-8b12-69a93dfd65d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:43.236736 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013bd0b-922f-4cd9-8b12-69a93dfd65d9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:43.246377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8702e-627c-4a35-a176-0befd9c2f636 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:43.256344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-1013bd0b-922f-4cd9-8b12-69a93dfd65d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 25 15:53:43.256344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:43.256344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:43.256344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:43.256344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:43.256344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:43.256344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:43.256344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:43.256344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:43.256344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:43.256344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:43.256344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:43.256344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:53:43.256344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:53:43.256344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:53:43.256344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:53:43.257412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:43.257412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:43.257412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:53:43.257412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:43.257412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:53:43.257412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:43.257412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:43.257412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:43.257412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:43.257412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:43.257412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:43.257412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:43.257412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:43.257412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:43.257412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:43.257412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:43.257412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:43.257412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:43.263653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:43.263653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:43.263653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:53:43.263653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:53:43.263653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:53:43.263653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:53:43.263653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:53:43.263653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:53:43.263653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 25 15:53:43.263653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:43.263653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 245/502] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Jun 25 15:53:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:43.263653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013bd0b-922f-4cd9-8b12-69a93dfd65d9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:43.282753 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542eaae-f528-4f56-acaf-f76f20818f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:43.283545 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542eaae-f528-4f56-acaf-f76f20818f16 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:43.283824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542eaae-f528-4f56-acaf-f76f20818f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:43.284096 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542eaae-f528-4f56-acaf-f76f20818f16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:43.286841 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:43.294874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542eaae-f528-4f56-acaf-f76f20818f16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:43.296360 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542eaae-f528-4f56-acaf-f76f20818f16 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:43.313801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-f542eaae-f528-4f56-acaf-f76f20818f16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:43.313801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:43.313801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:43.313801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:43.313801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:43.313801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:43.313801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:43.313801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:43.313801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:43.313801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:43.313801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:43.313801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:43.313801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:43.313801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:43.313801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:43.314678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:43.314678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:43.314678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:43.314678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:43.314678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:43.314678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:43.314678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:43.314678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:43.314678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:43.314678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:43.314678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:43.314678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:43.314678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:43.314678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:43.314678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:43.314678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:43.314678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:43.315570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:43.315570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:43.315570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:43.315570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:43.315570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 258/503] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:43.315856 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542eaae-f528-4f56-acaf-f76f20818f16 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:43.367452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013bd0b-922f-4cd9-8b12-69a93dfd65d9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:43.402910 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700cbf7a-ee6b-4cef-a2d5-eb681743c644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:43.403878 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cbf7a-ee6b-4cef-a2d5-eb681743c644 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:43.404160 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cbf7a-ee6b-4cef-a2d5-eb681743c644 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:43.404438 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cbf7a-ee6b-4cef-a2d5-eb681743c644 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:43.406751 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:43.412818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700cbf7a-ee6b-4cef-a2d5-eb681743c644 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:43.414461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700cbf7a-ee6b-4cef-a2d5-eb681743c644 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:43.415188 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542eaae-f528-4f56-acaf-f76f20818f16 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:43.429411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-700cbf7a-ee6b-4cef-a2d5-eb681743c644 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 25 15:53:43.429411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:43.429411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:43.429411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:43.429411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:43.429411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:43.429411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:43.429411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:43.429411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:43.429411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:43.429411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:43.429411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:43.429411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:53:43.429411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:53:43.429411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:53:43.429411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:53:43.430383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:43.430383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:43.430383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:53:43.430383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:43.430383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:53:43.430383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:43.430383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:43.430383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:43.430383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:43.430383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:43.430383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:43.430383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:43.430383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:43.430383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:43.430383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:43.430383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:43.430383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:43.430383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:43.431288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:43.431288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:43.431288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:53:43.431288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:53:43.431288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:53:43.431288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:53:43.431288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:53:43.431288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:53:43.431288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 25 15:53:43.431288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:43.431288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 246/504] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:43.431838 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700cbf7a-ee6b-4cef-a2d5-eb681743c644 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:43.453065 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a71962-52b8-46fc-b9fa-ab00da84301c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:43.453864 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a71962-52b8-46fc-b9fa-ab00da84301c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:43.454138 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a71962-52b8-46fc-b9fa-ab00da84301c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:43.454408 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a71962-52b8-46fc-b9fa-ab00da84301c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:43.464259 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a71962-52b8-46fc-b9fa-ab00da84301c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:43.471770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-d0a71962-52b8-46fc-b9fa-ab00da84301c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:43.471770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:43.471770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:43.471770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:43.471770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:43.471770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:43.471770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:43.471770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:43.471770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:43.471770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:43.471770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:43.471770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:43.471770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:43.471770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:43.471770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:43.471770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:43.472871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:43.472871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:43.472871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:43.472871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:43.472871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:43.472871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:43.472871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:43.472871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:43.472871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:43.472871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:43.472871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:43.472871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:43.472871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:43.472871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:43.472871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:43.472871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:43.473863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:43.473863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:43.473863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:43.473863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:43.473863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:43.473863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:43.473863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:43.473863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:43.473863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:43.473863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:43.473863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:43.473863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:43.473863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 259/505] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:53:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:43.473863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a71962-52b8-46fc-b9fa-ab00da84301c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:43.522555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:43.534572 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700cbf7a-ee6b-4cef-a2d5-eb681743c644 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:43.568039 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a9773-841f-4fd1-a3ba-bad76b104260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:43.568641 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a9773-841f-4fd1-a3ba-bad76b104260 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:43.568915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a9773-841f-4fd1-a3ba-bad76b104260 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:43.569181 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a9773-841f-4fd1-a3ba-bad76b104260 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:43.569900 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a9773-841f-4fd1-a3ba-bad76b104260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:43.570197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a9773-841f-4fd1-a3ba-bad76b104260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:43.571624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a9773-841f-4fd1-a3ba-bad76b104260 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:43.574025 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a71962-52b8-46fc-b9fa-ab00da84301c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:43.582589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 247/506] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:43.583351 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a9773-841f-4fd1-a3ba-bad76b104260 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:43.609951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3abec7-a1d5-4441-b500-2082d33b8965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:43.610620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3abec7-a1d5-4441-b500-2082d33b8965 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:43.610897 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3abec7-a1d5-4441-b500-2082d33b8965 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:43.611179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3abec7-a1d5-4441-b500-2082d33b8965 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:43.612024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3abec7-a1d5-4441-b500-2082d33b8965 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:43.612333 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3abec7-a1d5-4441-b500-2082d33b8965 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:43.613826 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3abec7-a1d5-4441-b500-2082d33b8965 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:43.637539 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:43.651534 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 260/507] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Jun 25 15:53:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:43.652534 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3abec7-a1d5-4441-b500-2082d33b8965 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:43.683603 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a9773-841f-4fd1-a3ba-bad76b104260 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:43.717536 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda00eaf-2855-4911-b8a7-cb75cd9bb595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:43.718187 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda00eaf-2855-4911-b8a7-cb75cd9bb595 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:43.718471 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda00eaf-2855-4911-b8a7-cb75cd9bb595 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:43.718735 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda00eaf-2855-4911-b8a7-cb75cd9bb595 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:43.719451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda00eaf-2855-4911-b8a7-cb75cd9bb595 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:43.719749 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda00eaf-2855-4911-b8a7-cb75cd9bb595 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:43.721140 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda00eaf-2855-4911-b8a7-cb75cd9bb595 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:43.732583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 248/508] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:43.733355 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda00eaf-2855-4911-b8a7-cb75cd9bb595 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:43.755579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3abec7-a1d5-4441-b500-2082d33b8965 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:43.763284 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:43.793445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d91c7-f6ed-4918-a1e1-0350fece5308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:43.794117 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d91c7-f6ed-4918-a1e1-0350fece5308 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:43.794400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d91c7-f6ed-4918-a1e1-0350fece5308 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:43.794669 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d91c7-f6ed-4918-a1e1-0350fece5308 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:43.795507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d91c7-f6ed-4918-a1e1-0350fece5308 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:43.795812 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d91c7-f6ed-4918-a1e1-0350fece5308 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:43.797283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d91c7-f6ed-4918-a1e1-0350fece5308 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:43.814118 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 261/509] 10.222.0.13 () {56 vars in 1137 bytes} [Sat Jun 25 15:53:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:43.814932 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d91c7-f6ed-4918-a1e1-0350fece5308 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:43.834180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda00eaf-2855-4911-b8a7-cb75cd9bb595 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:43.848576 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:43.870891 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dce363-c0c5-43ca-a483-31b999d9868a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:43.871576 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dce363-c0c5-43ca-a483-31b999d9868a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:43.871859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dce363-c0c5-43ca-a483-31b999d9868a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:43.872132 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dce363-c0c5-43ca-a483-31b999d9868a None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:43.872928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dce363-c0c5-43ca-a483-31b999d9868a None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:43.873234 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dce363-c0c5-43ca-a483-31b999d9868a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:43.874620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dce363-c0c5-43ca-a483-31b999d9868a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:43.882568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 249/510] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:53:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:43.883348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dce363-c0c5-43ca-a483-31b999d9868a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:43.918954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d91c7-f6ed-4918-a1e1-0350fece5308 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:43.951541 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484742f-3547-44d0-85d2-19130a8da371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:43.952328 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484742f-3547-44d0-85d2-19130a8da371 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:43.952636 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484742f-3547-44d0-85d2-19130a8da371 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:43.952902 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484742f-3547-44d0-85d2-19130a8da371 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:43.959751 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484742f-3547-44d0-85d2-19130a8da371 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:43.961233 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484742f-3547-44d0-85d2-19130a8da371 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:43.976635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-2484742f-3547-44d0-85d2-19130a8da371 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 25 15:53:43.976635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:43.976635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:43.976635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:43.976635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:43.976635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:43.976635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:43.976635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:43.976635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:43.976635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:43.976635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:43.976635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:43.976635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:53:43.976635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:53:43.976635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:53:43.976635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:53:43.977647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:43.977647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:43.977647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:53:43.977647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:43.977647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:53:43.977647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:43.977647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:43.977647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:43.977647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:43.977647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:43.977647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:43.977647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:43.977647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:43.977647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:43.977647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:43.977647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:43.977647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:43.977647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:43.978966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:43.978966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:43.978966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:53:43.978966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:53:43.978966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:53:43.978966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:53:43.978966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:53:43.978966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:53:43.978966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 25 15:53:43.978966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:43.978966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 262/511] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:43.979657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484742f-3547-44d0-85d2-19130a8da371 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:43.983887 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dce363-c0c5-43ca-a483-31b999d9868a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:44.006840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:44.022035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcfd71-7728-48ca-abba-7fc4914dca2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:44.022684 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcfd71-7728-48ca-abba-7fc4914dca2a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:44.022967 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcfd71-7728-48ca-abba-7fc4914dca2a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:44.023290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcfd71-7728-48ca-abba-7fc4914dca2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:44.068362 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:44.081851 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484742f-3547-44d0-85d2-19130a8da371 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:44.091166 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcfd71-7728-48ca-abba-7fc4914dca2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:44.092970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcfd71-7728-48ca-abba-7fc4914dca2a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:44.115310 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff16be-8688-42e6-893b-fcca5aa509ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:44.115985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff16be-8688-42e6-893b-fcca5aa509ce None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:44.116270 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff16be-8688-42e6-893b-fcca5aa509ce None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:44.116595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff16be-8688-42e6-893b-fcca5aa509ce None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:44.117310 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff16be-8688-42e6-893b-fcca5aa509ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:44.118901 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff16be-8688-42e6-893b-fcca5aa509ce None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:44.138991 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 263/512] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Jun 25 15:53:43 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:53:44.139806 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff16be-8688-42e6-893b-fcca5aa509ce None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:44.212834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 250/513] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:44.212834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcfd71-7728-48ca-abba-7fc4914dca2a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:44.239171 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:44.262840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff16be-8688-42e6-893b-fcca5aa509ce None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:44.299467 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:44.303526 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7e7a9-2335-4ded-a8ad-2cb855b9b62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:44.304184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7e7a9-2335-4ded-a8ad-2cb855b9b62a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:44.305022 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7e7a9-2335-4ded-a8ad-2cb855b9b62a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:44.305022 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7e7a9-2335-4ded-a8ad-2cb855b9b62a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:44.327894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcfd71-7728-48ca-abba-7fc4914dca2a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:44.367752 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882737c-41dc-4772-a7be-e17d9b83e11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:44.368946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882737c-41dc-4772-a7be-e17d9b83e11d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:44.369509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882737c-41dc-4772-a7be-e17d9b83e11d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:44.370242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882737c-41dc-4772-a7be-e17d9b83e11d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:44.379143 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b7e7a9-2335-4ded-a8ad-2cb855b9b62a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:44.380626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b7e7a9-2335-4ded-a8ad-2cb855b9b62a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:44.383041 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882737c-41dc-4772-a7be-e17d9b83e11d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:44.384691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882737c-41dc-4772-a7be-e17d9b83e11d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:44.403037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-5882737c-41dc-4772-a7be-e17d9b83e11d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:44.403037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:44.403037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:44.403037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:44.403037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:44.403037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:44.403037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:44.403037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:44.403037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:44.403037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:44.403037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:44.403037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:44.403037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:44.403037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:44.403037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:44.404087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:44.404087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:44.404087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:44.404087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:44.404087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:44.404087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:44.404087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:44.404087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:44.404087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:44.404087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:44.404087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:44.404087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:44.404087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:44.404087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:44.404087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:44.404087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:44.404087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:44.405108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:44.405108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:44.405108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:44.405108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:44.405108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 251/514] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:44.409603 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882737c-41dc-4772-a7be-e17d9b83e11d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:44.470586 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:44.491761 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 264/515] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:44.492731 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7e7a9-2335-4ded-a8ad-2cb855b9b62a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:44.515855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882737c-41dc-4772-a7be-e17d9b83e11d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:44.543902 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:44.553247 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1669743-564b-4031-99ff-e02a591c1902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:44.554060 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1669743-564b-4031-99ff-e02a591c1902 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:44.554340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1669743-564b-4031-99ff-e02a591c1902 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:44.554611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1669743-564b-4031-99ff-e02a591c1902 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:44.555454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1669743-564b-4031-99ff-e02a591c1902 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:44.555981 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1669743-564b-4031-99ff-e02a591c1902 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:44.557469 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1669743-564b-4031-99ff-e02a591c1902 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:44.597700 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 252/516] 10.222.0.13 () {56 vars in 1137 bytes} [Sat Jun 25 15:53:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:44.598509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1669743-564b-4031-99ff-e02a591c1902 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:44.603686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7e7a9-2335-4ded-a8ad-2cb855b9b62a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:44.640688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93788022-bfd8-4acd-ade6-45552151ca8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:44.642058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93788022-bfd8-4acd-ade6-45552151ca8a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:44.642340 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93788022-bfd8-4acd-ade6-45552151ca8a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:44.642915 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93788022-bfd8-4acd-ade6-45552151ca8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:44.654933 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93788022-bfd8-4acd-ade6-45552151ca8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:44.656394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93788022-bfd8-4acd-ade6-45552151ca8a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:44.674004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-93788022-bfd8-4acd-ade6-45552151ca8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:44.674004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:44.674004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:44.674004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:44.674004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:44.674004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:44.674004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:44.674004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:44.674004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:44.674004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:44.674004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:44.674004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:44.674004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:44.674004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:44.674004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:44.674871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:44.674871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:44.674871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:44.674871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:44.674871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:44.674871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:44.674871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:44.674871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:44.674871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:44.674871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:44.674871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:44.674871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:44.674871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:44.674871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:44.674871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:44.674871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:44.674871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:44.675738 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:44.675738 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:44.675738 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:44.675738 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:44.675738 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 265/517] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:44.675990 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93788022-bfd8-4acd-ade6-45552151ca8a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:44.705457 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1669743-564b-4031-99ff-e02a591c1902 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:44.708887 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:44.740484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71dae76-6e07-4efb-b64b-eef02bde14f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:44.741150 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dae76-6e07-4efb-b64b-eef02bde14f5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:44.741434 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dae76-6e07-4efb-b64b-eef02bde14f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:44.741705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71dae76-6e07-4efb-b64b-eef02bde14f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:44.742435 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71dae76-6e07-4efb-b64b-eef02bde14f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:44.742737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71dae76-6e07-4efb-b64b-eef02bde14f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:44.744297 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71dae76-6e07-4efb-b64b-eef02bde14f5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:44.760384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 253/518] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:44.760384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71dae76-6e07-4efb-b64b-eef02bde14f5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:44.785716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93788022-bfd8-4acd-ade6-45552151ca8a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:44.807358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:44.824461 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc8731-fed1-4618-9400-b023db294806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:44.825417 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc8731-fed1-4618-9400-b023db294806 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:44.825797 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc8731-fed1-4618-9400-b023db294806 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:44.826176 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc8731-fed1-4618-9400-b023db294806 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:44.836647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc8731-fed1-4618-9400-b023db294806 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:44.839866 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc8731-fed1-4618-9400-b023db294806 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:44.858627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-95bc8731-fed1-4618-9400-b023db294806 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:44.858627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:44.858627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:44.858627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:44.858627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:44.858627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:44.858627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:44.858627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:44.858627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:44.858627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:44.858627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:44.858627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:44.858627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:44.858627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:44.858627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:44.859694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:44.859694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:44.859694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:44.859694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:44.859694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:44.859694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:44.859694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:44.859694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:44.859694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:44.859694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:44.859694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:44.859694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:44.859694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:44.859694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:44.859694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:44.859694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:44.859694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:44.860751 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:44.860751 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:44.860751 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:44.860751 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:44.864839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 266/519] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:44.865284 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc8731-fed1-4618-9400-b023db294806 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:44.871507 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71dae76-6e07-4efb-b64b-eef02bde14f5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:44.909943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a8709-efad-4fe7-b1c9-62523c6374a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:44.910895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a8709-efad-4fe7-b1c9-62523c6374a9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:44.911190 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a8709-efad-4fe7-b1c9-62523c6374a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:44.911467 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a8709-efad-4fe7-b1c9-62523c6374a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:44.953470 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:44.969995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc8731-fed1-4618-9400-b023db294806 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:44.985414 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a8709-efad-4fe7-b1c9-62523c6374a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:44.986903 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a8709-efad-4fe7-b1c9-62523c6374a9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:45.003975 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f7bc2-08dc-4906-905e-ea9aad50a4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:45.004672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f7bc2-08dc-4906-905e-ea9aad50a4ee None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:45.004953 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f7bc2-08dc-4906-905e-ea9aad50a4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:45.005228 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f7bc2-08dc-4906-905e-ea9aad50a4ee None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:45.005949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f7bc2-08dc-4906-905e-ea9aad50a4ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:45.006252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f7bc2-08dc-4906-905e-ea9aad50a4ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:45.007626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f7bc2-08dc-4906-905e-ea9aad50a4ee None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:45.020139 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 267/520] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:45.020784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f7bc2-08dc-4906-905e-ea9aad50a4ee None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:45.039641 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:45.100947 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 254/521] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:45.100947 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a8709-efad-4fe7-b1c9-62523c6374a9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:45.129115 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f7bc2-08dc-4906-905e-ea9aad50a4ee None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:45.164338 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4709c7-58f0-4d78-bfcb-386c46129796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:45.165110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4709c7-58f0-4d78-bfcb-386c46129796 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:45.165440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4709c7-58f0-4d78-bfcb-386c46129796 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:45.165712 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4709c7-58f0-4d78-bfcb-386c46129796 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:45.173557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4709c7-58f0-4d78-bfcb-386c46129796 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:45.175121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4709c7-58f0-4d78-bfcb-386c46129796 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:45.190089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-8c4709c7-58f0-4d78-bfcb-386c46129796 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 25 15:53:45.190089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:45.190089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:45.190089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:45.190089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:45.190089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:45.190089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:45.190089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:45.190089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:45.190089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:45.190089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:45.190089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:45.190089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:53:45.190089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:53:45.190089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:53:45.190089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:53:45.191200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:45.191200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:45.191200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:53:45.191200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:45.191200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:53:45.191200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:45.191200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:45.191200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:45.191200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:45.191200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:45.191200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:45.191200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:45.191200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:45.191200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:45.191200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:45.191200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:45.191200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:45.191200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:45.192144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:45.192144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:45.192144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:53:45.192144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:53:45.192144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:53:45.192144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:53:45.192144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:53:45.192144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:53:45.192144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 25 15:53:45.192144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:45.192144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 268/522] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:53:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:45.192144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4709c7-58f0-4d78-bfcb-386c46129796 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:45.196297 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:45.203272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a8709-efad-4fe7-b1c9-62523c6374a9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:45.242877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ee8e7-d7d8-4594-bdf4-8739c5e3ddf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:45.243662 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ee8e7-d7d8-4594-bdf4-8739c5e3ddf9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:45.243877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ee8e7-d7d8-4594-bdf4-8739c5e3ddf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:45.244172 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ee8e7-d7d8-4594-bdf4-8739c5e3ddf9 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:45.244937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ee8e7-d7d8-4594-bdf4-8739c5e3ddf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:45.245235 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ee8e7-d7d8-4594-bdf4-8739c5e3ddf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:45.246626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ee8e7-d7d8-4594-bdf4-8739c5e3ddf9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:45.259064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 255/523] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:45.259885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ee8e7-d7d8-4594-bdf4-8739c5e3ddf9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:45.269428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:45.309033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4709c7-58f0-4d78-bfcb-386c46129796 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:45.344352 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2477cbf-6a2f-4b72-b2b4-c456d99773d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:45.345180 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2477cbf-6a2f-4b72-b2b4-c456d99773d6 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:45.345502 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2477cbf-6a2f-4b72-b2b4-c456d99773d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:45.346031 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2477cbf-6a2f-4b72-b2b4-c456d99773d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:45.353374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2477cbf-6a2f-4b72-b2b4-c456d99773d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:45.354847 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2477cbf-6a2f-4b72-b2b4-c456d99773d6 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:45.366875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ee8e7-d7d8-4594-bdf4-8739c5e3ddf9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:45.369699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-e2477cbf-6a2f-4b72-b2b4-c456d99773d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:45.369699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:45.369699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:45.369699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:45.369699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:45.369699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:45.369699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:45.369699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:45.369699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:45.369699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:45.369699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:45.369699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:45.369699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:53:45.369699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:53:45.369699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:53:45.370721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:53:45.370721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:45.370721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:45.370721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:45.370721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:45.370721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:45.370721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:45.370721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:45.370721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:45.370721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:45.370721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:45.370721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:45.370721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:45.370721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:45.370721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:45.370721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:45.370721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:45.376663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:53:45.376663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:53:45.376663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:53:45.376663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:53:45.376663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:53:45.376663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:53:45.376663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:45.376663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:45.376663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 269/524] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:45.376663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2477cbf-6a2f-4b72-b2b4-c456d99773d6 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:45.411262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f645e-72ad-4873-8017-c87504eeeeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:45.412053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f645e-72ad-4873-8017-c87504eeeeee None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:45.412375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f645e-72ad-4873-8017-c87504eeeeee None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:45.412689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f645e-72ad-4873-8017-c87504eeeeee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:45.424691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f645e-72ad-4873-8017-c87504eeeeee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:45.426555 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f645e-72ad-4873-8017-c87504eeeeee None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:45.437555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:45.446321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-f00f645e-72ad-4873-8017-c87504eeeeee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:45.446321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:45.446321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:45.446321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:45.446321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:45.446321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:45.446321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:45.446321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:45.446321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:45.446321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:45.446321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:45.446321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:45.446321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:45.446321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:45.446321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:45.452394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:45.452394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:45.452394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:45.452394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:45.452394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:45.452394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:45.452394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:45.452394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:45.452394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:45.452394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:45.452394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:45.452394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:45.452394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:45.452394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:45.452394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:45.452394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:45.452394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:45.453500 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:45.453500 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:45.453500 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:45.453500 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:45.453500 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 256/525] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:45.453500 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f645e-72ad-4873-8017-c87504eeeeee None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:45.485949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2477cbf-6a2f-4b72-b2b4-c456d99773d6 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:45.512450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:45.523065 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf04459-8bf7-42b0-88da-a00f6bcebed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:45.523751 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf04459-8bf7-42b0-88da-a00f6bcebed8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:45.524025 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf04459-8bf7-42b0-88da-a00f6bcebed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:45.524373 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf04459-8bf7-42b0-88da-a00f6bcebed8 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:45.525301 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf04459-8bf7-42b0-88da-a00f6bcebed8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:45.525617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf04459-8bf7-42b0-88da-a00f6bcebed8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:45.527051 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf04459-8bf7-42b0-88da-a00f6bcebed8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:45.544704 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 270/526] 10.222.0.13 () {56 vars in 1139 bytes} [Sat Jun 25 15:53:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:45.545348 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf04459-8bf7-42b0-88da-a00f6bcebed8 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:45.564113 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f645e-72ad-4873-8017-c87504eeeeee None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:45.600856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c003408-1564-4f59-b727-09797a6b9836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:45.601921 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c003408-1564-4f59-b727-09797a6b9836 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:45.602345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c003408-1564-4f59-b727-09797a6b9836 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:45.603258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c003408-1564-4f59-b727-09797a6b9836 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:45.610475 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c003408-1564-4f59-b727-09797a6b9836 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:45.612010 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c003408-1564-4f59-b727-09797a6b9836 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:45.628769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-0c003408-1564-4f59-b727-09797a6b9836 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:45.628769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:45.628769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:45.628769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:45.628769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:45.628769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:45.628769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:45.628769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:45.628769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:45.628769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:45.628769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:45.628769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:45.628769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:53:45.628769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:53:45.628769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:53:45.629816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:53:45.629816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:45.629816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:45.629816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:45.629816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:45.629816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:45.629816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:45.629816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:45.629816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:45.629816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:45.629816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:45.629816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:45.629816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:45.629816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:45.629816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:45.629816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:45.629816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:45.630839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:53:45.630839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:53:45.630839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:53:45.630839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:53:45.630839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:53:45.630839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:53:45.630839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:45.630839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:45.637788 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 257/527] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:45.637788 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c003408-1564-4f59-b727-09797a6b9836 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:45.655737 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf04459-8bf7-42b0-88da-a00f6bcebed8 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:45.679965 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:45.694859 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f86d1-57bf-4396-bcf8-44c94f3a3d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:45.695623 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f86d1-57bf-4396-bcf8-44c94f3a3d8f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:45.695974 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f86d1-57bf-4396-bcf8-44c94f3a3d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:45.696454 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f86d1-57bf-4396-bcf8-44c94f3a3d8f None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:45.697284 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f86d1-57bf-4396-bcf8-44c94f3a3d8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:45.697592 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f86d1-57bf-4396-bcf8-44c94f3a3d8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:45.699040 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f86d1-57bf-4396-bcf8-44c94f3a3d8f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:45.722720 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 271/528] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:53:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:45.722720 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f86d1-57bf-4396-bcf8-44c94f3a3d8f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:45.752218 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c003408-1564-4f59-b727-09797a6b9836 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:45.770128 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:45.787749 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce21f5-2783-493e-9da5-c15021c0d2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:45.788428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce21f5-2783-493e-9da5-c15021c0d2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:45.788726 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce21f5-2783-493e-9da5-c15021c0d2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:45.789001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce21f5-2783-493e-9da5-c15021c0d2a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:45.789722 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce21f5-2783-493e-9da5-c15021c0d2a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:45.790053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce21f5-2783-493e-9da5-c15021c0d2a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:45.791433 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce21f5-2783-493e-9da5-c15021c0d2a5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:45.803223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 258/529] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:45.804024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce21f5-2783-493e-9da5-c15021c0d2a5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:45.840075 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f86d1-57bf-4396-bcf8-44c94f3a3d8f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:45.881185 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d233cea-052f-474e-bf55-09e0ce150d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:45.881845 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d233cea-052f-474e-bf55-09e0ce150d49 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:45.882128 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d233cea-052f-474e-bf55-09e0ce150d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:45.882411 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d233cea-052f-474e-bf55-09e0ce150d49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:45.916729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce21f5-2783-493e-9da5-c15021c0d2a5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:45.935895 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:45.952293 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e99952-08e1-4c5e-8d20-43ce6eaffc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:45.953104 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e99952-08e1-4c5e-8d20-43ce6eaffc66 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:45.953390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e99952-08e1-4c5e-8d20-43ce6eaffc66 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:45.953665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e99952-08e1-4c5e-8d20-43ce6eaffc66 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:45.954521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e99952-08e1-4c5e-8d20-43ce6eaffc66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:45.954880 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e99952-08e1-4c5e-8d20-43ce6eaffc66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:45.956237 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d233cea-052f-474e-bf55-09e0ce150d49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:45.956342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e99952-08e1-4c5e-8d20-43ce6eaffc66 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:45.957699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d233cea-052f-474e-bf55-09e0ce150d49 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:45.975457 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 259/530] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:53:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:45.976283 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e99952-08e1-4c5e-8d20-43ce6eaffc66 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:46.021523 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:46.082379 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 272/531] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:46.082379 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d233cea-052f-474e-bf55-09e0ce150d49 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:46.092223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e99952-08e1-4c5e-8d20-43ce6eaffc66 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:46.138951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a14c8-1822-4118-b4c1-bdb34bf82948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:46.139766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a14c8-1822-4118-b4c1-bdb34bf82948 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:46.140323 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a14c8-1822-4118-b4c1-bdb34bf82948 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:46.140643 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a14c8-1822-4118-b4c1-bdb34bf82948 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:46.142083 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a14c8-1822-4118-b4c1-bdb34bf82948 None admin] RBAC: Adding query filter params (role.id=7f1996bcf7e54638b4f53b3d8ba83495, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=3622c5ccb7534e7c8e8342b4f6643bc3) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:46.142399 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a14c8-1822-4118-b4c1-bdb34bf82948 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:46.144003 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a14c8-1822-4118-b4c1-bdb34bf82948 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:46.150853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 260/532] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:53:45 2022] GET /identity/v3/role_assignments?user.id=3622c5ccb7534e7c8e8342b4f6643bc3&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=7f1996bcf7e54638b4f53b3d8ba83495 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:46.151647 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a14c8-1822-4118-b4c1-bdb34bf82948 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:46.170600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:46.193253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d233cea-052f-474e-bf55-09e0ce150d49 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:46.233446 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dfe4a1-c063-4ef5-98ce-383cf907dabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:46.234261 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfe4a1-c063-4ef5-98ce-383cf907dabd None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:46.234542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfe4a1-c063-4ef5-98ce-383cf907dabd None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:46.234816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfe4a1-c063-4ef5-98ce-383cf907dabd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:46.242632 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dfe4a1-c063-4ef5-98ce-383cf907dabd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:46.244285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dfe4a1-c063-4ef5-98ce-383cf907dabd None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:46.259840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-54dfe4a1-c063-4ef5-98ce-383cf907dabd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:46.259840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:46.259840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:46.259840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:46.259840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:46.259840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:46.259840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:46.259840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:46.259840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:46.259840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:46.259840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:46.259840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:46.259840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:53:46.259840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:53:46.259840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:53:46.260877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:53:46.260877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:46.260877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:46.260877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:46.260877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:46.260877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:46.260877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:46.260877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:46.260877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:46.260877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:46.260877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:46.260877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:46.260877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:46.260877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:46.260877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:46.260877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:46.260877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:46.266914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:53:46.266914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:53:46.266914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:53:46.266914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:53:46.266914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:53:46.266914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:53:46.266914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:46.266914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:46.266914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 273/533] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:46.266914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dfe4a1-c063-4ef5-98ce-383cf907dabd None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:46.281939 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a14c8-1822-4118-b4c1-bdb34bf82948 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:46.283911 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:46.321711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96ffed-7ead-4f12-bc0a-2d9fd1e79fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:46.322324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96ffed-7ead-4f12-bc0a-2d9fd1e79fff None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:46.322607 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96ffed-7ead-4f12-bc0a-2d9fd1e79fff None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:46.322881 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96ffed-7ead-4f12-bc0a-2d9fd1e79fff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:46.324718 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd96ffed-7ead-4f12-bc0a-2d9fd1e79fff None admin] RBAC: Adding query filter params (role.id=7f1996bcf7e54638b4f53b3d8ba83495, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=5cba086728824ec68fb0ee90fb3f99dd) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:46.324718 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd96ffed-7ead-4f12-bc0a-2d9fd1e79fff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:46.325477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd96ffed-7ead-4f12-bc0a-2d9fd1e79fff None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:46.331805 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 261/534] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:53:46 2022] GET /identity/v3/role_assignments?user.id=5cba086728824ec68fb0ee90fb3f99dd&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=7f1996bcf7e54638b4f53b3d8ba83495 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:46.333122 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c3fe9-6b58-48d6-9b71-a92b1b80ae21 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:46.333345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c3fe9-6b58-48d6-9b71-a92b1b80ae21 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:46.333620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c3fe9-6b58-48d6-9b71-a92b1b80ae21 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:46.334972 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 262/535] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:46.335550 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c3fe9-6b58-48d6-9b71-a92b1b80ae21 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:46.395012 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dfe4a1-c063-4ef5-98ce-383cf907dabd None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:46.440383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928a618-fd0b-441e-b922-c4c9cd52ac0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:46.441687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928a618-fd0b-441e-b922-c4c9cd52ac0f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:46.441979 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928a618-fd0b-441e-b922-c4c9cd52ac0f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:46.442252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928a618-fd0b-441e-b922-c4c9cd52ac0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:46.458424 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928a618-fd0b-441e-b922-c4c9cd52ac0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:46.459832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9928a618-fd0b-441e-b922-c4c9cd52ac0f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:46.460294 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c3fe9-6b58-48d6-9b71-a92b1b80ae21 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:46.462113 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:46.483952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-9928a618-fd0b-441e-b922-c4c9cd52ac0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:46.483952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:46.483952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:46.483952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:46.483952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:46.483952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:46.483952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:46.483952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:46.483952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:46.483952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:46.483952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:46.483952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:46.483952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:46.483952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:46.483952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:46.485101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:46.485101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:46.485101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:46.485101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:46.485101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:46.485101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:46.485101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:46.485101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:46.485101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:46.485101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:46.485101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:46.485101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:46.485101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:46.485101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:46.485101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:46.485101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:46.485101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:46.486134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:46.486134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:46.486134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:46.486134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:46.488545 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 274/536] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:46.490066 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a2460-995a-4561-bf74-53a71dc13c71 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:46.490486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a2460-995a-4561-bf74-53a71dc13c71 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:46.490881 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a2460-995a-4561-bf74-53a71dc13c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:46.508668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-807a2460-995a-4561-bf74-53a71dc13c71 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:46.510208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-807a2460-995a-4561-bf74-53a71dc13c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:46.513817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf1243-0c5c-4c31-bb45-b63be27f4037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:46.515328 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf1243-0c5c-4c31-bb45-b63be27f4037 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:46.515611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf1243-0c5c-4c31-bb45-b63be27f4037 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:46.515872 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf1243-0c5c-4c31-bb45-b63be27f4037 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:46.517032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf1243-0c5c-4c31-bb45-b63be27f4037 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:46.517338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf1243-0c5c-4c31-bb45-b63be27f4037 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:46.518797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf1243-0c5c-4c31-bb45-b63be27f4037 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:46.541109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 263/537] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:53:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:46.542084 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf1243-0c5c-4c31-bb45-b63be27f4037 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:46.576630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:46.604555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 275/538] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:46.605589 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a2460-995a-4561-bf74-53a71dc13c71 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:46.673326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf1243-0c5c-4c31-bb45-b63be27f4037 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:46.722911 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1bee9-40ba-487b-b489-2ec599721251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:46.723624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1bee9-40ba-487b-b489-2ec599721251 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:46.724176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1bee9-40ba-487b-b489-2ec599721251 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:46.724550 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1bee9-40ba-487b-b489-2ec599721251 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:46.725811 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1bee9-40ba-487b-b489-2ec599721251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:46.726207 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1bee9-40ba-487b-b489-2ec599721251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:46.727961 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1bee9-40ba-487b-b489-2ec599721251 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:46.744372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 264/539] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:46.748923 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1bee9-40ba-487b-b489-2ec599721251 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:46.749033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a2460-995a-4561-bf74-53a71dc13c71 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:46.780874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:46.796381 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164313e6-adbd-4eb5-ac03-5aa114087d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:46.797206 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164313e6-adbd-4eb5-ac03-5aa114087d94 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:46.797488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164313e6-adbd-4eb5-ac03-5aa114087d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:46.797765 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164313e6-adbd-4eb5-ac03-5aa114087d94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:46.798669 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164313e6-adbd-4eb5-ac03-5aa114087d94 None admin] RBAC: Adding query filter params (role.id=7f1996bcf7e54638b4f53b3d8ba83495, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=b00152c72a1f4755b09c1dbd1d874b11) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:46.798986 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164313e6-adbd-4eb5-ac03-5aa114087d94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:46.801067 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164313e6-adbd-4eb5-ac03-5aa114087d94 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:46.816544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 276/540] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:53:46 2022] GET /identity/v3/role_assignments?user.id=b00152c72a1f4755b09c1dbd1d874b11&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=7f1996bcf7e54638b4f53b3d8ba83495 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:46.816544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164313e6-adbd-4eb5-ac03-5aa114087d94 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:46.889001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:46.919330 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1bee9-40ba-487b-b489-2ec599721251 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:46.988217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b871d48-6d85-4ba5-b8da-ac7238520591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:46.989169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b871d48-6d85-4ba5-b8da-ac7238520591 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:46.989448 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b871d48-6d85-4ba5-b8da-ac7238520591 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:46.989749 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b871d48-6d85-4ba5-b8da-ac7238520591 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:47.009234 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b871d48-6d85-4ba5-b8da-ac7238520591 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:47.017959 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164313e6-adbd-4eb5-ac03-5aa114087d94 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:47.019258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-9b871d48-6d85-4ba5-b8da-ac7238520591 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:47.019258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:47.019258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:47.019258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:47.019258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:47.019258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:47.019258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:47.019258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:47.019258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:47.019258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:47.019258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:47.019258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:47.019258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:47.019258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:47.019258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:47.019258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:47.020159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:47.020159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:47.020159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:47.020159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:47.020159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:47.020159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:47.020159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:47.020159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:47.020159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:47.020159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:47.020159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:47.020159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:47.020159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:47.020159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:47.020159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:47.020159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:47.020992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:47.020992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:47.020992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:47.020992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:47.020992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:47.020992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:47.020992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:47.020992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:47.020992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:47.020992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:47.020992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:47.020992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:47.024661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 265/541] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:53:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:47.036089 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b871d48-6d85-4ba5-b8da-ac7238520591 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:47.082020 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56371c59-387d-44aa-b487-24bb540809e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:47.082824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56371c59-387d-44aa-b487-24bb540809e3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:47.083104 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56371c59-387d-44aa-b487-24bb540809e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:47.083378 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56371c59-387d-44aa-b487-24bb540809e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:47.099753 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56371c59-387d-44aa-b487-24bb540809e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:47.101253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56371c59-387d-44aa-b487-24bb540809e3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:47.128688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-56371c59-387d-44aa-b487-24bb540809e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:47.128688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:47.128688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:47.128688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:47.128688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:47.128688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:47.128688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:47.128688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:47.128688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:47.128688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:47.128688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:47.128688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:47.128688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:53:47.128688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:53:47.128688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:53:47.129601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:53:47.129601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:47.129601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:47.129601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:47.129601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:47.129601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:47.129601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:47.129601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:47.129601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:47.129601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:47.129601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:47.129601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:47.129601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:47.129601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:47.129601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:47.129601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:47.129601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:47.130450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:53:47.130450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:53:47.130450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:53:47.130450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:53:47.130450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:53:47.130450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:53:47.130450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:47.130450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:47.136841 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 277/542] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:47.136841 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56371c59-387d-44aa-b487-24bb540809e3 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:47.232635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b871d48-6d85-4ba5-b8da-ac7238520591 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:47.236783 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:47.295603 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56371c59-387d-44aa-b487-24bb540809e3 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:47.321249 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29e20c-a7a7-405c-bc13-1023fa2bf4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:47.322559 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29e20c-a7a7-405c-bc13-1023fa2bf4ab None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:47.322844 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29e20c-a7a7-405c-bc13-1023fa2bf4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:47.323120 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd29e20c-a7a7-405c-bc13-1023fa2bf4ab None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:47.323893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29e20c-a7a7-405c-bc13-1023fa2bf4ab None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:47.324204 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29e20c-a7a7-405c-bc13-1023fa2bf4ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:47.325650 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd29e20c-a7a7-405c-bc13-1023fa2bf4ab None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:47.335157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b130401-8fbd-4aba-8b19-898bb9d4fee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:47.335850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b130401-8fbd-4aba-8b19-898bb9d4fee7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:47.336130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b130401-8fbd-4aba-8b19-898bb9d4fee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:47.336421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b130401-8fbd-4aba-8b19-898bb9d4fee7 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:47.338113 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b130401-8fbd-4aba-8b19-898bb9d4fee7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:47.338320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b130401-8fbd-4aba-8b19-898bb9d4fee7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:47.339801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b130401-8fbd-4aba-8b19-898bb9d4fee7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:47.345040 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 266/543] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:53:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:47.350493 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29e20c-a7a7-405c-bc13-1023fa2bf4ab None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:47.366074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 278/544] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:53:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:47.368759 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:47.377742 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b130401-8fbd-4aba-8b19-898bb9d4fee7 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:47.534394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b130401-8fbd-4aba-8b19-898bb9d4fee7 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:47.546632 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:47.564337 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd29e20c-a7a7-405c-bc13-1023fa2bf4ab None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:47.589836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a7432-efff-4321-9b20-6293c8dd8f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:47.590501 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a7432-efff-4321-9b20-6293c8dd8f28 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:47.590782 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a7432-efff-4321-9b20-6293c8dd8f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:47.591055 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a7432-efff-4321-9b20-6293c8dd8f28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:47.591896 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a7432-efff-4321-9b20-6293c8dd8f28 None admin] RBAC: Adding query filter params (role.id=7f1996bcf7e54638b4f53b3d8ba83495, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=aaffb9f0bf714822829455c0487376ca) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:47.592207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a7432-efff-4321-9b20-6293c8dd8f28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:47.593685 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a7432-efff-4321-9b20-6293c8dd8f28 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:47.605760 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 279/545] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:53:47 2022] GET /identity/v3/role_assignments?user.id=aaffb9f0bf714822829455c0487376ca&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=7f1996bcf7e54638b4f53b3d8ba83495 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:47.625393 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b82399-2595-43ae-bbb5-ee027c3559aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:47.626236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b82399-2595-43ae-bbb5-ee027c3559aa None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:47.626518 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b82399-2595-43ae-bbb5-ee027c3559aa None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:47.626791 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b82399-2595-43ae-bbb5-ee027c3559aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:47.639956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b82399-2595-43ae-bbb5-ee027c3559aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:47.641909 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b82399-2595-43ae-bbb5-ee027c3559aa None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:47.660664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-60b82399-2595-43ae-bbb5-ee027c3559aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:47.660664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:47.660664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:47.660664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:47.660664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:47.660664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:47.660664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:47.660664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:47.660664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:47.660664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:47.660664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:47.660664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:47.660664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:47.660664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:47.660664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:47.661620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:47.661620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:47.661620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:47.661620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:47.661620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:47.661620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:47.661620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:47.661620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:47.661620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:47.661620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:47.661620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:47.661620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:47.661620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:47.661620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:47.661620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:47.661620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:47.661620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:47.662466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:47.662466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:47.662466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:47.662466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:47.662466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 267/546] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:47.669588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a7432-efff-4321-9b20-6293c8dd8f28 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:47.847486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a7432-efff-4321-9b20-6293c8dd8f28 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:47.894545 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ad8f4-82fa-4c95-a2fd-847dd7c71636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:47.895219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ad8f4-82fa-4c95-a2fd-847dd7c71636 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:47.895501 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ad8f4-82fa-4c95-a2fd-847dd7c71636 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:47.895775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ad8f4-82fa-4c95-a2fd-847dd7c71636 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:47.896535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ad8f4-82fa-4c95-a2fd-847dd7c71636 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:47.896844 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ad8f4-82fa-4c95-a2fd-847dd7c71636 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:47.898226 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ad8f4-82fa-4c95-a2fd-847dd7c71636 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:47.917741 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 280/547] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:47.922862 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b82399-2595-43ae-bbb5-ee027c3559aa None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:48.005533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:48.051784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b82399-2595-43ae-bbb5-ee027c3559aa None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:48.085554 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32537ca-bf67-41de-b73a-04af7b81053f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:48.086352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32537ca-bf67-41de-b73a-04af7b81053f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:48.086634 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32537ca-bf67-41de-b73a-04af7b81053f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:48.087075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32537ca-bf67-41de-b73a-04af7b81053f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:48.095968 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32537ca-bf67-41de-b73a-04af7b81053f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:48.097545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32537ca-bf67-41de-b73a-04af7b81053f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:48.111977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-f32537ca-bf67-41de-b73a-04af7b81053f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 25 15:53:48.111977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:48.111977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:48.111977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:48.111977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:48.111977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:48.111977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:48.111977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:48.111977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:48.111977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:48.111977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:48.111977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:48.111977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:53:48.111977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:53:48.111977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:53:48.111977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:53:48.113045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:48.113045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:48.113045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:53:48.113045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:48.113045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:53:48.113045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:48.113045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:48.113045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:48.113045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:48.113045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:48.113045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:48.113045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:48.113045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:48.113045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:48.113045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:48.113045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:48.113045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:48.113045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:48.114407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:48.114407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:48.114407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:53:48.114407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:53:48.114407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:53:48.114407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:53:48.114407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:53:48.114407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:53:48.114407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 25 15:53:48.114407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:48.114407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 268/548] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:53:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:48.121573 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ad8f4-82fa-4c95-a2fd-847dd7c71636 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:48.165965 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:48.259472 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ad8f4-82fa-4c95-a2fd-847dd7c71636 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:48.307583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c0a46-130e-4004-8475-128401a1887a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:48.308269 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c0a46-130e-4004-8475-128401a1887a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:48.308577 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c0a46-130e-4004-8475-128401a1887a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:48.308853 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c0a46-130e-4004-8475-128401a1887a None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:48.309695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c0a46-130e-4004-8475-128401a1887a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:48.310001 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c0a46-130e-4004-8475-128401a1887a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:48.311430 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c0a46-130e-4004-8475-128401a1887a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:48.329931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 281/549] 10.222.0.13 () {56 vars in 1135 bytes} [Sat Jun 25 15:53:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:48.338012 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32537ca-bf67-41de-b73a-04af7b81053f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:48.431526 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556710f-e662-4830-bbf1-bff370258a01 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:48.431836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556710f-e662-4830-bbf1-bff370258a01 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:48.432104 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556710f-e662-4830-bbf1-bff370258a01 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:48.433781 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 282/550] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:53:48.443337 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c84ac-376f-4374-a205-63bd287d8a74 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:48.443627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c84ac-376f-4374-a205-63bd287d8a74 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:48.443910 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c84ac-376f-4374-a205-63bd287d8a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:48.461170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f80c84ac-376f-4374-a205-63bd287d8a74 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:48.462506 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f80c84ac-376f-4374-a205-63bd287d8a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:48.492120 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32537ca-bf67-41de-b73a-04af7b81053f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:48.533399 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:48.557305 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 283/551] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:53:48.561314 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81a78d-9c4b-4ce4-8e00-4b0889ca83d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:48.562037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81a78d-9c4b-4ce4-8e00-4b0889ca83d9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:48.562318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81a78d-9c4b-4ce4-8e00-4b0889ca83d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:48.562595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81a78d-9c4b-4ce4-8e00-4b0889ca83d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:48.601522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c84ac-376f-4374-a205-63bd287d8a74 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:48.634227 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:48.675872 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81a78d-9c4b-4ce4-8e00-4b0889ca83d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:48.679343 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81a78d-9c4b-4ce4-8e00-4b0889ca83d9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:48.763417 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c84ac-376f-4374-a205-63bd287d8a74 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:48.797335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 269/552] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:48.798692 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc5498-d8ae-4fb4-a566-f953f308b72d None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:48.798980 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc5498-d8ae-4fb4-a566-f953f308b72d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:48.799256 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc5498-d8ae-4fb4-a566-f953f308b72d None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:48.804761 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 270/553] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:48.805722 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc5498-d8ae-4fb4-a566-f953f308b72d None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:48.812226 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9ef36-acaf-4b90-af2f-eb0373f218f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:48.813085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9ef36-acaf-4b90-af2f-eb0373f218f9 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:48.813373 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9ef36-acaf-4b90-af2f-eb0373f218f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:48.813647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9ef36-acaf-4b90-af2f-eb0373f218f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:48.827795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc9ef36-acaf-4b90-af2f-eb0373f218f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:48.829037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:48.840661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-7dc9ef36-acaf-4b90-af2f-eb0373f218f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:48.840661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:48.840661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:48.840661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:48.840661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:48.840661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:48.840661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:48.840661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:48.840661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:48.840661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:48.840661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:48.840661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:48.840661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:48.840661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:48.840661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:48.840661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:48.841730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:48.841730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:48.841730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:48.841730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:48.841730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:48.841730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:48.841730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:48.841730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:48.841730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:48.841730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:48.841730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:48.841730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:48.841730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:48.841730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:48.841730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:48.841730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:48.842549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:48.842549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:48.842549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:48.842549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:48.842549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:48.842549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:48.842549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:48.842549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:48.842549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:48.842549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:48.842549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:48.842549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:48.842549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 284/554] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:53:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:48.846968 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da818357-c5a5-41fa-b9c9-a9b229de8706 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:48.846968 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da818357-c5a5-41fa-b9c9-a9b229de8706 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:48.846968 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da818357-c5a5-41fa-b9c9-a9b229de8706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:48.869769 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-da818357-c5a5-41fa-b9c9-a9b229de8706 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:48.871418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-da818357-c5a5-41fa-b9c9-a9b229de8706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:48.928555 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc5498-d8ae-4fb4-a566-f953f308b72d None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:48.956436 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 285/555] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:48.957444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da818357-c5a5-41fa-b9c9-a9b229de8706 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:48.965805 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f928d-1a9e-41be-bafd-052e9fc21ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:48.966609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f928d-1a9e-41be-bafd-052e9fc21ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:48.966890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f928d-1a9e-41be-bafd-052e9fc21ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:48.967162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f928d-1a9e-41be-bafd-052e9fc21ca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:48.978534 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f928d-1a9e-41be-bafd-052e9fc21ca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:48.980009 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f928d-1a9e-41be-bafd-052e9fc21ca3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:48.982326 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:49.000766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-fc0f928d-1a9e-41be-bafd-052e9fc21ca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:49.000766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:49.000766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:49.000766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:49.000766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:49.000766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:49.000766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:49.000766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:49.000766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:49.000766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:49.000766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:49.000766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:49.000766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:49.000766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:49.000766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:49.001822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:49.001822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:49.001822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:49.001822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:49.001822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:49.001822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:49.001822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:49.001822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:49.001822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:49.001822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:49.001822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:49.001822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:49.001822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:49.001822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:49.001822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:49.001822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:49.001822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:49.002694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:49.002694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:49.002694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:49.002694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:49.002694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 271/556] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:49.003548 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f928d-1a9e-41be-bafd-052e9fc21ca3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:49.082503 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:49.115682 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f928d-1a9e-41be-bafd-052e9fc21ca3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:49.116746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da818357-c5a5-41fa-b9c9-a9b229de8706 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:49.153094 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ac13d-4d01-4c5f-a457-f287c087d8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:49.153899 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ac13d-4d01-4c5f-a457-f287c087d8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:49.154178 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ac13d-4d01-4c5f-a457-f287c087d8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:49.154450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ac13d-4d01-4c5f-a457-f287c087d8c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:49.165999 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b767093-50ef-4806-ab8e-6fa092e08333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:49.166509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ac13d-4d01-4c5f-a457-f287c087d8c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:49.167428 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b767093-50ef-4806-ab8e-6fa092e08333 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:49.167911 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b767093-50ef-4806-ab8e-6fa092e08333 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:49.168425 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b767093-50ef-4806-ab8e-6fa092e08333 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:49.169403 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b767093-50ef-4806-ab8e-6fa092e08333 None admin] RBAC: Adding query filter params (name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:49.169910 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b767093-50ef-4806-ab8e-6fa092e08333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:49.171514 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b767093-50ef-4806-ab8e-6fa092e08333 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:49.174333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-a55ac13d-4d01-4c5f-a457-f287c087d8c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:49.174333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:49.174333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:49.174333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:49.174333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:49.174333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:49.174333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:49.174333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:49.174333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:49.174333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:49.174333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:49.174333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:49.174333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:49.174333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:49.174333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:49.174333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:49.175727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:49.175727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:49.175727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:49.175727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:49.175727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:49.175727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:49.175727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:49.175727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:49.175727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:49.175727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:49.175727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:49.175727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:49.175727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:49.175727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:49.175727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:49.175727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:49.176617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:49.176617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:49.176617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:49.176617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:49.176617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:49.176617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:49.176617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:49.176617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:49.176617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:49.176617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:49.176617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:49.176617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:49.181772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 272/557] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:53:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:49.182465 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ac13d-4d01-4c5f-a457-f287c087d8c0 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:49.185540 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 286/558] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:53:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:49.186909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b767093-50ef-4806-ab8e-6fa092e08333 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:49.282464 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:49.304642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ac13d-4d01-4c5f-a457-f287c087d8c0 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:49.318405 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b767093-50ef-4806-ab8e-6fa092e08333 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:49.339830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:49.346041 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef8b1a-4510-4a24-9042-da8471aac891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:49.346981 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef8b1a-4510-4a24-9042-da8471aac891 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:49.347267 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef8b1a-4510-4a24-9042-da8471aac891 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:49.347668 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef8b1a-4510-4a24-9042-da8471aac891 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:49.348795 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef8b1a-4510-4a24-9042-da8471aac891 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:49.349201 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef8b1a-4510-4a24-9042-da8471aac891 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:49.350499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef8b1a-4510-4a24-9042-da8471aac891 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:49.358063 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1765c-b21a-4ca1-8288-7078984540dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:49.359019 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1765c-b21a-4ca1-8288-7078984540dd None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:49.359482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1765c-b21a-4ca1-8288-7078984540dd None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:49.360381 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1765c-b21a-4ca1-8288-7078984540dd None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:49.361652 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff1765c-b21a-4ca1-8288-7078984540dd None admin] RBAC: Adding query filter params (name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:49.362163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff1765c-b21a-4ca1-8288-7078984540dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:49.363072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 273/559] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:49.363892 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef8b1a-4510-4a24-9042-da8471aac891 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:49.364051 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff1765c-b21a-4ca1-8288-7078984540dd None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:49.375457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 287/560] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:53:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:49.376800 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e98d13-64ae-46c8-918a-ec07cf21a80a None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:49.377092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e98d13-64ae-46c8-918a-ec07cf21a80a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:49.377371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e98d13-64ae-46c8-918a-ec07cf21a80a None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:49.380547 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 288/561] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:49.381540 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e98d13-64ae-46c8-918a-ec07cf21a80a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:49.476016 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef8b1a-4510-4a24-9042-da8471aac891 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:49.491705 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e98d13-64ae-46c8-918a-ec07cf21a80a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:49.512146 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6f183-767d-400a-b920-3ef7d806a49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:49.512971 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6f183-767d-400a-b920-3ef7d806a49c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:49.513247 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6f183-767d-400a-b920-3ef7d806a49c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:49.513533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6f183-767d-400a-b920-3ef7d806a49c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:49.525021 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6f183-767d-400a-b920-3ef7d806a49c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:49.526636 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6f183-767d-400a-b920-3ef7d806a49c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:49.531158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514d552-ff05-4fc1-a687-bb7e88d9c7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:49.531957 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514d552-ff05-4fc1-a687-bb7e88d9c7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:49.532239 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514d552-ff05-4fc1-a687-bb7e88d9c7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:49.532602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514d552-ff05-4fc1-a687-bb7e88d9c7e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:49.534629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:49.540801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514d552-ff05-4fc1-a687-bb7e88d9c7e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:49.542931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514d552-ff05-4fc1-a687-bb7e88d9c7e6 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:49.547714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-32f6f183-767d-400a-b920-3ef7d806a49c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:49.547714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:49.547714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:49.547714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:49.547714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:49.547714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:49.547714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:49.547714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:49.547714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:49.547714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:49.547714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:49.547714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:49.547714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:49.547714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:49.547714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:49.548834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:49.548834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:49.548834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:49.548834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:49.548834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:49.548834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:49.548834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:49.548834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:49.548834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:49.548834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:49.548834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:49.548834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:49.548834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:49.548834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:49.548834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:49.548834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:49.548834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:49.549754 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:49.549754 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:49.549754 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:49.549754 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:49.552719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 274/562] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:49.553757 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6f183-767d-400a-b920-3ef7d806a49c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:49.558600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-d514d552-ff05-4fc1-a687-bb7e88d9c7e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:49.558600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:49.558600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:49.558600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:49.558600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:49.558600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:49.558600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:49.558600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:49.558600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:49.558600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:49.558600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:49.558600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:49.558600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:53:49.558600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:53:49.558600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:53:49.559463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:53:49.559463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:49.559463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:49.559463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:49.559463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:49.559463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:49.559463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:49.559463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:49.559463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:49.559463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:49.559463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:49.559463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:49.559463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:49.559463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:49.559463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:49.559463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:49.559463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:49.560341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:53:49.560341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:53:49.560341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:53:49.560341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:53:49.560341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:53:49.560341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:53:49.560341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:49.560341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:49.564470 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 289/563] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:49.565746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7759c-83f4-41b8-89db-de1f68536b2f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:49.566036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7759c-83f4-41b8-89db-de1f68536b2f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:49.566317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7759c-83f4-41b8-89db-de1f68536b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:49.583725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-daa7759c-83f4-41b8-89db-de1f68536b2f None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:49.584963 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-daa7759c-83f4-41b8-89db-de1f68536b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:49.606033 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:49.656342 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 290/564] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:49.657223 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7759c-83f4-41b8-89db-de1f68536b2f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:49.674016 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6f183-767d-400a-b920-3ef7d806a49c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:49.711828 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd86dcb-3077-4ba9-a876-c3f7b4078c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:49.713577 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd86dcb-3077-4ba9-a876-c3f7b4078c2b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:49.714049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd86dcb-3077-4ba9-a876-c3f7b4078c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:49.714381 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd86dcb-3077-4ba9-a876-c3f7b4078c2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:49.724479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd86dcb-3077-4ba9-a876-c3f7b4078c2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:49.725994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd86dcb-3077-4ba9-a876-c3f7b4078c2b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:49.744723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-edd86dcb-3077-4ba9-a876-c3f7b4078c2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:49.744723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:49.744723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:49.744723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:49.744723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:49.744723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:49.744723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:49.744723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:49.744723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:49.744723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:49.744723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:49.744723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:49.744723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:49.744723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:49.744723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:49.745863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:49.745863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:49.745863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:49.745863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:49.745863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:49.745863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:49.745863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:49.745863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:49.745863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:49.745863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:49.745863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:49.745863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:49.745863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:49.745863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:49.745863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:49.745863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:49.745863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:49.746985 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:49.746985 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:49.746985 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:49.746985 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:49.747669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 275/565] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:49.752191 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd86dcb-3077-4ba9-a876-c3f7b4078c2b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:49.770043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7759c-83f4-41b8-89db-de1f68536b2f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:49.799727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:49.814811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d02332-780a-44a9-8a9a-2e2131f24e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:49.815604 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d02332-780a-44a9-8a9a-2e2131f24e94 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:49.815977 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d02332-780a-44a9-8a9a-2e2131f24e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:49.816356 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d02332-780a-44a9-8a9a-2e2131f24e94 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:49.817201 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d02332-780a-44a9-8a9a-2e2131f24e94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:49.817599 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d02332-780a-44a9-8a9a-2e2131f24e94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:49.819062 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d02332-780a-44a9-8a9a-2e2131f24e94 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:49.836895 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 291/566] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:49.836895 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d02332-780a-44a9-8a9a-2e2131f24e94 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:49.859448 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:49.866772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd86dcb-3077-4ba9-a876-c3f7b4078c2b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:49.901442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109ce70-e70c-4c1f-a9db-040c0808dcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:49.902107 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109ce70-e70c-4c1f-a9db-040c0808dcc1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:49.902383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109ce70-e70c-4c1f-a9db-040c0808dcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:49.902648 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109ce70-e70c-4c1f-a9db-040c0808dcc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:49.903433 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109ce70-e70c-4c1f-a9db-040c0808dcc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:49.903735 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109ce70-e70c-4c1f-a9db-040c0808dcc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:49.905193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109ce70-e70c-4c1f-a9db-040c0808dcc1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:49.918487 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 276/567] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:53:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:49.919121 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109ce70-e70c-4c1f-a9db-040c0808dcc1 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:49.943271 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d02332-780a-44a9-8a9a-2e2131f24e94 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:49.975609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcf9c1-4bf6-46a0-b6ff-f40f881a8fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:49.976400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcf9c1-4bf6-46a0-b6ff-f40f881a8fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:49.976711 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcf9c1-4bf6-46a0-b6ff-f40f881a8fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:49.976974 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcf9c1-4bf6-46a0-b6ff-f40f881a8fd4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:49.986856 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcf9c1-4bf6-46a0-b6ff-f40f881a8fd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:49.994640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-f0fcf9c1-4bf6-46a0-b6ff-f40f881a8fd4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:49.994640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:49.994640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:49.994640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:49.994640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:49.994640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:49.994640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:49.994640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:49.994640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:49.994640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:49.994640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:49.994640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:49.994640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:49.994640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:49.994640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:49.994640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:49.995554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:49.995554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:49.995554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:49.995554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:49.995554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:49.995554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:49.995554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:49.995554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:49.995554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:49.995554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:49.995554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:49.995554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:49.995554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:49.995554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:49.995554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:49.995554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:49.996376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:49.996376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:49.996376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:49.996376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:49.996376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:49.996376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:49.996376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:49.996376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:49.996376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:49.996376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:49.996376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:49.996376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:49.996376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 292/568] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:53:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:49.997103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcf9c1-4bf6-46a0-b6ff-f40f881a8fd4 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:50.021518 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109ce70-e70c-4c1f-a9db-040c0808dcc1 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:50.046057 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:50.055854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a8b43-2bce-4d4f-b247-61957bdc67b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:50.056546 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a8b43-2bce-4d4f-b247-61957bdc67b2 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:50.056827 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a8b43-2bce-4d4f-b247-61957bdc67b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:50.057102 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a8b43-2bce-4d4f-b247-61957bdc67b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:50.057832 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a8b43-2bce-4d4f-b247-61957bdc67b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:50.058137 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a8b43-2bce-4d4f-b247-61957bdc67b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:50.059540 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a8b43-2bce-4d4f-b247-61957bdc67b2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:50.072847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 277/569] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:50.073645 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a8b43-2bce-4d4f-b247-61957bdc67b2 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:50.093490 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:50.104598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcf9c1-4bf6-46a0-b6ff-f40f881a8fd4 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:50.138559 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9a1a3-f37a-458f-bad7-74f62b29504d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:50.139573 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9a1a3-f37a-458f-bad7-74f62b29504d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:50.140019 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9a1a3-f37a-458f-bad7-74f62b29504d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:50.140437 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9a1a3-f37a-458f-bad7-74f62b29504d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:50.148818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9a1a3-f37a-458f-bad7-74f62b29504d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:50.150494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e9a1a3-f37a-458f-bad7-74f62b29504d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:50.164280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-d6e9a1a3-f37a-458f-bad7-74f62b29504d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 25 15:53:50.164280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:50.164280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:50.164280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:50.164280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:50.164280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:50.164280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:50.164280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:50.164280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:50.164280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:50.164280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:50.164280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:50.164280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:53:50.164280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:53:50.164280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:53:50.164280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:53:50.165492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:50.165492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:50.165492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:53:50.165492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:50.165492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:53:50.165492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:50.165492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:50.165492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:50.165492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:50.165492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:50.165492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:50.165492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:50.165492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:50.165492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:50.165492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:50.165492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:50.165492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:50.165492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:50.166940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:50.166940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:50.166940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:53:50.166940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:53:50.166940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:53:50.166940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:53:50.166940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:53:50.166940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:53:50.166940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 25 15:53:50.166940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:50.166940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 293/570] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Jun 25 15:53:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:50.166940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20c370-2cbc-49cf-883a-bba97a1aced8 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:50.166940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20c370-2cbc-49cf-883a-bba97a1aced8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:50.167688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20c370-2cbc-49cf-883a-bba97a1aced8 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:50.168469 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 294/571] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:50.169663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20c370-2cbc-49cf-883a-bba97a1aced8 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:50.180991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a8b43-2bce-4d4f-b247-61957bdc67b2 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:50.217694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035c7eb-b0d6-436d-8d26-8ead132cff21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:50.218343 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035c7eb-b0d6-436d-8d26-8ead132cff21 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:50.218623 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035c7eb-b0d6-436d-8d26-8ead132cff21 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:50.218893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035c7eb-b0d6-436d-8d26-8ead132cff21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:50.219725 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035c7eb-b0d6-436d-8d26-8ead132cff21 None admin] RBAC: Adding query filter params (role.id=7f1996bcf7e54638b4f53b3d8ba83495, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=19788c4239ca44f9a31a7d32086c8947) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:50.220026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035c7eb-b0d6-436d-8d26-8ead132cff21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:50.221481 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035c7eb-b0d6-436d-8d26-8ead132cff21 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:50.226835 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 278/572] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:53:50 2022] GET /identity/v3/role_assignments?user.id=19788c4239ca44f9a31a7d32086c8947&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=7f1996bcf7e54638b4f53b3d8ba83495 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:50.227635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035c7eb-b0d6-436d-8d26-8ead132cff21 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:50.279872 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20c370-2cbc-49cf-883a-bba97a1aced8 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:50.283589 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:50.320820 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07eb90-c471-4585-ab83-b305337ecf41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:50.321545 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07eb90-c471-4585-ab83-b305337ecf41 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:50.321824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07eb90-c471-4585-ab83-b305337ecf41 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:50.322097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07eb90-c471-4585-ab83-b305337ecf41 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:50.322969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07eb90-c471-4585-ab83-b305337ecf41 None admin] RBAC: Adding query filter params (name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:50.323972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07eb90-c471-4585-ab83-b305337ecf41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:50.324811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07eb90-c471-4585-ab83-b305337ecf41 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:50.338877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 295/573] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:53:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:50.339085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:50.339180 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07eb90-c471-4585-ab83-b305337ecf41 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:50.361861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035c7eb-b0d6-436d-8d26-8ead132cff21 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:50.412650 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19f220-27c0-474b-924c-a1d09200d9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:50.413481 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19f220-27c0-474b-924c-a1d09200d9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:50.413759 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19f220-27c0-474b-924c-a1d09200d9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:50.414375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19f220-27c0-474b-924c-a1d09200d9e3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:50.422881 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa19f220-27c0-474b-924c-a1d09200d9e3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:50.424329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa19f220-27c0-474b-924c-a1d09200d9e3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:50.443842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-fa19f220-27c0-474b-924c-a1d09200d9e3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 25 15:53:50.443842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:50.443842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:50.443842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:50.443842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:50.443842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:50.443842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:50.443842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:50.443842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:50.443842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:50.443842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:50.443842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:50.443842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:53:50.443842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:53:50.443842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:53:50.443842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:53:50.444843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:50.444843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:50.444843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:53:50.444843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:50.444843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:53:50.444843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:50.444843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:50.444843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:50.444843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:50.444843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:50.444843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:50.444843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:50.444843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:50.444843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:50.444843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:50.444843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:50.444843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:50.444843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:50.445779 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:50.445779 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:50.445779 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:53:50.445779 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:53:50.445779 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:53:50.445779 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:53:50.445779 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:53:50.445779 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:53:50.445779 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 25 15:53:50.445779 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:50.447163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 279/574] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:50.448692 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02fc75-a8cb-48a9-8f69-4da7493bc001 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:50.449050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02fc75-a8cb-48a9-8f69-4da7493bc001 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:50.449386 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d02fc75-a8cb-48a9-8f69-4da7493bc001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:50.452762 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07eb90-c471-4585-ab83-b305337ecf41 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:50.466436 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-7d02fc75-a8cb-48a9-8f69-4da7493bc001 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:50.467640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7d02fc75-a8cb-48a9-8f69-4da7493bc001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:50.489175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4818b-7d3a-4d4c-8101-03e560510282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:50.489848 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4818b-7d3a-4d4c-8101-03e560510282 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:50.490133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4818b-7d3a-4d4c-8101-03e560510282 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:50.490411 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4818b-7d3a-4d4c-8101-03e560510282 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:50.491253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4818b-7d3a-4d4c-8101-03e560510282 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:50.491564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4818b-7d3a-4d4c-8101-03e560510282 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:50.493102 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4818b-7d3a-4d4c-8101-03e560510282 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:50.511971 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 296/575] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Jun 25 15:53:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:50.512844 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4818b-7d3a-4d4c-8101-03e560510282 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:50.539258 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:50.552685 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 280/576] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:50.553470 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02fc75-a8cb-48a9-8f69-4da7493bc001 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:50.623737 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4818b-7d3a-4d4c-8101-03e560510282 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:50.625704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:50.660569 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d02fc75-a8cb-48a9-8f69-4da7493bc001 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:50.662457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c3fc9-97f9-46c2-bf3d-e882a5033334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:50.663321 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c3fc9-97f9-46c2-bf3d-e882a5033334 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:50.663539 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c3fc9-97f9-46c2-bf3d-e882a5033334 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:50.663812 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c3fc9-97f9-46c2-bf3d-e882a5033334 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:50.674696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c3fc9-97f9-46c2-bf3d-e882a5033334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:50.676183 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c3fc9-97f9-46c2-bf3d-e882a5033334 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:50.694398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-404c3fc9-97f9-46c2-bf3d-e882a5033334 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:50.694398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:50.694398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:50.694398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:50.694398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:50.694398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:50.694398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:50.694398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:50.694398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:50.694398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:50.694398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:50.694398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:50.694398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:50.694398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:50.694398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:50.695313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:50.695313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:50.695313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:50.695313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:50.695313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:50.695313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:50.695313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:50.695313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:50.695313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:50.695313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:50.695313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:50.695313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:50.695313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:50.695313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:50.695313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:50.695313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:50.695313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:50.696530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:50.696530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:50.696530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:50.696530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:50.696530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 297/577] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:50.696821 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c3fc9-97f9-46c2-bf3d-e882a5033334 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:50.700894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc29d8c-cb0f-4bc3-a0dc-0292bb97997f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:50.701862 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc29d8c-cb0f-4bc3-a0dc-0292bb97997f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:50.702275 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc29d8c-cb0f-4bc3-a0dc-0292bb97997f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:50.702798 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc29d8c-cb0f-4bc3-a0dc-0292bb97997f None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:50.704151 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc29d8c-cb0f-4bc3-a0dc-0292bb97997f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:50.704479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc29d8c-cb0f-4bc3-a0dc-0292bb97997f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:50.706390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc29d8c-cb0f-4bc3-a0dc-0292bb97997f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:50.730280 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 281/578] 10.222.0.13 () {56 vars in 1137 bytes} [Sat Jun 25 15:53:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:50.730280 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc29d8c-cb0f-4bc3-a0dc-0292bb97997f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:50.778374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:50.804836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c3fc9-97f9-46c2-bf3d-e882a5033334 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:50.837624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc29d8c-cb0f-4bc3-a0dc-0292bb97997f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:50.844884 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600c77c-2474-4023-bf5d-1dbf20b5efdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:50.845691 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600c77c-2474-4023-bf5d-1dbf20b5efdf None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:50.845971 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600c77c-2474-4023-bf5d-1dbf20b5efdf None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:50.846247 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600c77c-2474-4023-bf5d-1dbf20b5efdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:50.859062 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600c77c-2474-4023-bf5d-1dbf20b5efdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:50.859062 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600c77c-2474-4023-bf5d-1dbf20b5efdf None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:50.867434 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:50.876229 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d9abb-946f-4581-9a97-4c8ac5948a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:50.877323 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d9abb-946f-4581-9a97-4c8ac5948a61 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:50.877800 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d9abb-946f-4581-9a97-4c8ac5948a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:50.878198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-d600c77c-2474-4023-bf5d-1dbf20b5efdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:50.878198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:50.878198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:50.878198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:50.878198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:50.878198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:50.878198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:50.878198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:50.878198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:50.878198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:50.878198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:50.878198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:50.878198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:50.878198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:50.878198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:50.879071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:50.879071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:50.879071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:50.879071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:50.879071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:50.879071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:50.879071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:50.879071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:50.879071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:50.879071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:50.879071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:50.879071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:50.879071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:50.879071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:50.879071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:50.879071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:50.879071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:50.879958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:50.879958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:50.879958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:50.879958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:50.879958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 298/579] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:50.880228 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600c77c-2474-4023-bf5d-1dbf20b5efdf None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:50.880855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d9abb-946f-4581-9a97-4c8ac5948a61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:50.896203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d9abb-946f-4581-9a97-4c8ac5948a61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:50.904075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-431d9abb-946f-4581-9a97-4c8ac5948a61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:50.904075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:50.904075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:50.904075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:50.904075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:50.904075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:50.904075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:50.904075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:50.904075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:50.904075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:50.904075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:50.904075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:50.904075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:50.904075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:50.904075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:50.904075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:50.905023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:50.905023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:50.905023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:50.905023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:50.905023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:50.905023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:50.905023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:50.905023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:50.905023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:50.905023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:50.905023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:50.905023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:50.905023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:50.905023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:50.905023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:50.905023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:50.905854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:50.905854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:50.905854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:50.905854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:50.905854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:50.905854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:50.905854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:50.905854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:50.905854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:50.905854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:50.905854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:50.905854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:50.905854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 282/580] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:53:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:50.906540 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d9abb-946f-4581-9a97-4c8ac5948a61 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:50.989951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600c77c-2474-4023-bf5d-1dbf20b5efdf None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:51.014500 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d9abb-946f-4581-9a97-4c8ac5948a61 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:51.018218 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:51.026647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d0cce-14e0-4be1-babe-fa69302fd91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:51.027312 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d0cce-14e0-4be1-babe-fa69302fd91a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:51.027594 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d0cce-14e0-4be1-babe-fa69302fd91a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:51.027870 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d0cce-14e0-4be1-babe-fa69302fd91a None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:51.028641 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d0cce-14e0-4be1-babe-fa69302fd91a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:51.028939 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d0cce-14e0-4be1-babe-fa69302fd91a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:51.030349 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d0cce-14e0-4be1-babe-fa69302fd91a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:51.042719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 299/581] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:51.045680 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d0cce-14e0-4be1-babe-fa69302fd91a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:51.049844 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb58d98-598e-419e-a2a3-9c7eec72367e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:51.050640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb58d98-598e-419e-a2a3-9c7eec72367e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:51.050920 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb58d98-598e-419e-a2a3-9c7eec72367e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:51.051192 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb58d98-598e-419e-a2a3-9c7eec72367e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:51.062172 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb58d98-598e-419e-a2a3-9c7eec72367e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:51.063648 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb58d98-598e-419e-a2a3-9c7eec72367e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:51.080521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-9bb58d98-598e-419e-a2a3-9c7eec72367e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:51.080521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:51.080521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:51.080521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:51.080521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:51.080521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:51.080521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:51.080521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:51.080521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:51.080521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:51.080521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:51.080521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:51.080521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:51.080521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:51.080521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:51.081609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:51.081609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:51.081609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:51.081609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:51.081609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:51.081609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:51.081609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:51.081609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:51.081609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:51.081609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:51.081609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:51.081609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:51.081609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:51.081609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:51.081609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:51.081609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:51.081609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:51.082499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:51.082499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:51.082499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:51.082499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:51.082499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 283/582] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:51.082499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb58d98-598e-419e-a2a3-9c7eec72367e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:51.133852 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:51.156194 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d0cce-14e0-4be1-babe-fa69302fd91a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:51.191248 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb391e8-c222-41e5-85c7-1703406f0710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:51.192058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb391e8-c222-41e5-85c7-1703406f0710 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:51.192346 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb391e8-c222-41e5-85c7-1703406f0710 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:51.192639 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb391e8-c222-41e5-85c7-1703406f0710 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:51.193369 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb391e8-c222-41e5-85c7-1703406f0710 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:51.193679 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb391e8-c222-41e5-85c7-1703406f0710 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:51.195047 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb391e8-c222-41e5-85c7-1703406f0710 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:51.195870 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb58d98-598e-419e-a2a3-9c7eec72367e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:51.207318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 300/583] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:51.209544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb391e8-c222-41e5-85c7-1703406f0710 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:51.231977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757efed-3645-4a1c-9eff-9fa8a2524bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:51.232734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757efed-3645-4a1c-9eff-9fa8a2524bfd None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:51.233016 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757efed-3645-4a1c-9eff-9fa8a2524bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:51.233332 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757efed-3645-4a1c-9eff-9fa8a2524bfd None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:51.234136 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757efed-3645-4a1c-9eff-9fa8a2524bfd None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:51.234501 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757efed-3645-4a1c-9eff-9fa8a2524bfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:51.235920 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757efed-3645-4a1c-9eff-9fa8a2524bfd None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:51.248621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 284/584] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:53:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:51.249468 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757efed-3645-4a1c-9eff-9fa8a2524bfd None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:51.268768 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:51.330183 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb391e8-c222-41e5-85c7-1703406f0710 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:51.361695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757efed-3645-4a1c-9eff-9fa8a2524bfd None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:51.363494 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:51.369238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aec4ef-5e83-4634-b8c8-e1d659b7938b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:51.370032 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aec4ef-5e83-4634-b8c8-e1d659b7938b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:51.370621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aec4ef-5e83-4634-b8c8-e1d659b7938b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:51.370621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aec4ef-5e83-4634-b8c8-e1d659b7938b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:51.378211 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aec4ef-5e83-4634-b8c8-e1d659b7938b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:51.379666 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aec4ef-5e83-4634-b8c8-e1d659b7938b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:51.394566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-c4aec4ef-5e83-4634-b8c8-e1d659b7938b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 25 15:53:51.394566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:51.394566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:51.394566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:51.394566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:51.394566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:51.394566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:51.394566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:51.394566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:51.394566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:51.394566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:51.394566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:51.394566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:53:51.394566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:53:51.394566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:53:51.394566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:53:51.395520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:51.395520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:51.395520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:53:51.395520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:51.395520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:53:51.395520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:51.395520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:51.395520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:51.395520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:51.395520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:51.395520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:51.395520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:51.395520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:51.395520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:51.395520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:51.395520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:51.395520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:51.395520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:51.396503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:51.396503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:51.396503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:53:51.396503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:53:51.396503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:53:51.396503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:53:51.396503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:53:51.396503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:53:51.396503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 25 15:53:51.396503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:51.396503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 301/585] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:51.397104 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aec4ef-5e83-4634-b8c8-e1d659b7938b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:51.405014 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2e550-1311-4e5e-9db1-4f4ed11c9532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:51.405656 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2e550-1311-4e5e-9db1-4f4ed11c9532 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:51.405937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2e550-1311-4e5e-9db1-4f4ed11c9532 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:51.406246 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2e550-1311-4e5e-9db1-4f4ed11c9532 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:51.406958 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2e550-1311-4e5e-9db1-4f4ed11c9532 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:51.407263 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2e550-1311-4e5e-9db1-4f4ed11c9532 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:51.408665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2e550-1311-4e5e-9db1-4f4ed11c9532 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:51.420862 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 285/586] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:51.421663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2e550-1311-4e5e-9db1-4f4ed11c9532 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:51.510589 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aec4ef-5e83-4634-b8c8-e1d659b7938b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:51.521278 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:51.523923 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2e550-1311-4e5e-9db1-4f4ed11c9532 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:51.549040 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a5ad3-f266-45d7-9ee4-756311dff0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:51.549898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a5ad3-f266-45d7-9ee4-756311dff0dd None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:51.550116 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a5ad3-f266-45d7-9ee4-756311dff0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:51.550391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a5ad3-f266-45d7-9ee4-756311dff0dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:51.557723 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a5ad3-f266-45d7-9ee4-756311dff0dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:51.559164 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a5ad3-f266-45d7-9ee4-756311dff0dd None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:51.559561 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccbb01-0073-4f4d-b828-ebde7a5525f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:51.560351 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccbb01-0073-4f4d-b828-ebde7a5525f7 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:51.560659 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccbb01-0073-4f4d-b828-ebde7a5525f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:51.560933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccbb01-0073-4f4d-b828-ebde7a5525f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:51.571285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccbb01-0073-4f4d-b828-ebde7a5525f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:51.572752 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccbb01-0073-4f4d-b828-ebde7a5525f7 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:51.573309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-257a5ad3-f266-45d7-9ee4-756311dff0dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:51.573309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:51.573309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:51.573309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:51.573309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:51.573309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:51.573309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:51.573309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:51.573309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:51.573309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:51.573309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:51.573309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:51.573309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:53:51.573309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:53:51.573309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:53:51.574126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:53:51.574126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:51.574126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:51.574126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:51.574126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:51.574126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:51.574126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:51.574126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:51.574126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:51.574126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:51.574126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:51.574126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:51.574126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:51.574126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:51.574126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:51.574126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:51.574126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:51.575028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:53:51.575028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:53:51.575028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:53:51.575028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:53:51.575028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:53:51.575028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:53:51.575028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:51.575028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:51.575028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 302/587] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:51.575505 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a5ad3-f266-45d7-9ee4-756311dff0dd None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:51.595611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-12ccbb01-0073-4f4d-b828-ebde7a5525f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:51.595611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:51.595611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:51.595611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:51.595611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:51.595611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:51.595611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:51.595611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:51.595611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:51.595611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:51.595611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:51.595611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:51.595611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:51.595611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:51.595611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:51.596615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:51.596615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:51.596615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:51.596615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:51.596615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:51.596615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:51.596615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:51.596615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:51.596615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:51.596615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:51.596615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:51.596615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:51.596615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:51.596615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:51.596615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:51.596615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:51.596615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:51.597484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:51.597484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:51.597484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:51.597484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:51.601477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 286/588] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:51.602409 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccbb01-0073-4f4d-b828-ebde7a5525f7 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:51.618654 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:51.683483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a5ad3-f266-45d7-9ee4-756311dff0dd None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:51.711536 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccbb01-0073-4f4d-b828-ebde7a5525f7 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:51.721047 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ac335-3e42-4a7d-a22d-567c41de9cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:51.721726 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ac335-3e42-4a7d-a22d-567c41de9cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:51.722005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ac335-3e42-4a7d-a22d-567c41de9cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:51.722275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ac335-3e42-4a7d-a22d-567c41de9cb3 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:51.723146 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ac335-3e42-4a7d-a22d-567c41de9cb3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:51.723451 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ac335-3e42-4a7d-a22d-567c41de9cb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:51.724931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ac335-3e42-4a7d-a22d-567c41de9cb3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:51.743129 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 303/589] 10.222.0.13 () {56 vars in 1137 bytes} [Sat Jun 25 15:53:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:51.748021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ac335-3e42-4a7d-a22d-567c41de9cb3 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:51.748467 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e4315-d83d-418d-952b-bf2ce9f3c02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:51.749271 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e4315-d83d-418d-952b-bf2ce9f3c02b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:51.749552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e4315-d83d-418d-952b-bf2ce9f3c02b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:51.749826 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e4315-d83d-418d-952b-bf2ce9f3c02b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:51.757505 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e4315-d83d-418d-952b-bf2ce9f3c02b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:51.759305 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e4315-d83d-418d-952b-bf2ce9f3c02b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:51.774492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-403e4315-d83d-418d-952b-bf2ce9f3c02b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:51.774492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:51.774492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:51.774492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:51.774492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:51.774492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:51.774492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:51.774492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:51.774492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:51.774492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:51.774492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:51.774492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:51.774492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:53:51.774492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:53:51.774492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:53:51.775524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:53:51.775524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:51.775524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:51.775524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:51.775524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:51.775524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:51.775524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:51.775524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:51.775524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:51.775524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:51.775524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:51.775524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:51.775524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:51.775524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:51.775524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:51.775524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:51.775524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:51.776767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:53:51.776767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:53:51.776767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:53:51.776767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:53:51.776767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:53:51.776767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:53:51.776767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:51.776767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:51.776767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 287/590] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:51.776767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e4315-d83d-418d-952b-bf2ce9f3c02b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:51.782067 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:51.855004 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:51.870924 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ac335-3e42-4a7d-a22d-567c41de9cb3 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:51.883977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e4315-d83d-418d-952b-bf2ce9f3c02b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:51.909283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab9d947-0640-4199-b146-1d1f459105c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:51.910221 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9d947-0640-4199-b146-1d1f459105c5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:51.910686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9d947-0640-4199-b146-1d1f459105c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:51.911039 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9d947-0640-4199-b146-1d1f459105c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:51.911739 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab9d947-0640-4199-b146-1d1f459105c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:51.912043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab9d947-0640-4199-b146-1d1f459105c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:51.913497 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab9d947-0640-4199-b146-1d1f459105c5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:51.918108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566ca67-a07b-4db2-aa08-891fd70706b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:51.918799 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566ca67-a07b-4db2-aa08-891fd70706b2 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:51.919078 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566ca67-a07b-4db2-aa08-891fd70706b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:51.919350 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566ca67-a07b-4db2-aa08-891fd70706b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:51.920088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566ca67-a07b-4db2-aa08-891fd70706b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:51.920391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566ca67-a07b-4db2-aa08-891fd70706b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:51.921810 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566ca67-a07b-4db2-aa08-891fd70706b2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:51.930599 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 304/591] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:53:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:51.930599 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab9d947-0640-4199-b146-1d1f459105c5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:51.940709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 288/592] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:51.941760 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566ca67-a07b-4db2-aa08-891fd70706b2 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:52.022304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:52.037770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab9d947-0640-4199-b146-1d1f459105c5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:52.054261 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566ca67-a07b-4db2-aa08-891fd70706b2 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:52.073044 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e049b99-5841-427b-be13-2ed8f8953fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:52.073848 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e049b99-5841-427b-be13-2ed8f8953fea None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:52.074131 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e049b99-5841-427b-be13-2ed8f8953fea None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:52.074408 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e049b99-5841-427b-be13-2ed8f8953fea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:52.085548 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e049b99-5841-427b-be13-2ed8f8953fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:52.087343 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e049b99-5841-427b-be13-2ed8f8953fea None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:52.092606 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0277399-7654-4ca2-bfdb-21164bf85fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:52.093273 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0277399-7654-4ca2-bfdb-21164bf85fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:52.093550 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0277399-7654-4ca2-bfdb-21164bf85fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:52.094115 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0277399-7654-4ca2-bfdb-21164bf85fa9 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:52.094611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0277399-7654-4ca2-bfdb-21164bf85fa9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:52.094914 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0277399-7654-4ca2-bfdb-21164bf85fa9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:52.096330 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0277399-7654-4ca2-bfdb-21164bf85fa9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:52.098176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:52.105179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-3e049b99-5841-427b-be13-2ed8f8953fea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:52.105179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:52.105179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:52.105179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:52.105179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:52.105179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:52.105179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:52.105179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:52.105179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:52.105179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:52.105179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:52.105179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:52.105179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:52.105179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:52.105179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:52.106219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:52.106219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:52.106219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:52.106219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:52.106219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:52.106219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:52.106219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:52.106219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:52.106219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:52.106219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:52.106219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:52.106219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:52.106219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:52.106219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:52.106219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:52.106219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:52.106219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:52.107134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:52.107134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:52.107134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:52.107134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:52.107134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 305/593] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:52.107478 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e049b99-5841-427b-be13-2ed8f8953fea None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:52.114466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 289/594] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:53:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:52.115452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0277399-7654-4ca2-bfdb-21164bf85fa9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:52.219502 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e049b99-5841-427b-be13-2ed8f8953fea None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:52.222631 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0277399-7654-4ca2-bfdb-21164bf85fa9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:52.255571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32e40b-f664-46cf-961a-406fe27cb1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:52.256485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32e40b-f664-46cf-961a-406fe27cb1ec None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:52.256762 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32e40b-f664-46cf-961a-406fe27cb1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:52.257032 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32e40b-f664-46cf-961a-406fe27cb1ec None admin] PATH_INFO: `/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/3622c5ccb7534e7c8e8342b4f6643bc3/roles/7f1996bcf7e54638b4f53b3d8ba83495` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:52.258320 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7a3ce-3c2d-4b5a-bbd3-d9bf7f39fc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:52.259720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7a3ce-3c2d-4b5a-bbd3-d9bf7f39fc80 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:52.260006 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7a3ce-3c2d-4b5a-bbd3-d9bf7f39fc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:52.260285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7a3ce-3c2d-4b5a-bbd3-d9bf7f39fc80 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:52.266128 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:52.268217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b7a3ce-3c2d-4b5a-bbd3-d9bf7f39fc80 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:52.269723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b7a3ce-3c2d-4b5a-bbd3-d9bf7f39fc80 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:52.273349 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db32e40b-f664-46cf-961a-406fe27cb1ec None admin] RBAC: Authorizing `identity:create_grant(project_id=3600648eb94b434b8c797d2f454e46b8, user_id=3622c5ccb7534e7c8e8342b4f6643bc3, role_id=7f1996bcf7e54638b4f53b3d8ba83495)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:52.274825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db32e40b-f664-46cf-961a-406fe27cb1ec None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:52.284047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-70b7a3ce-3c2d-4b5a-bbd3-d9bf7f39fc80 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 25 15:53:52.284047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:52.284047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:52.284047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:52.284047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:52.284047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:52.284047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:52.284047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:52.284047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:52.284047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:52.284047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:52.284047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:52.284047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:53:52.284047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:53:52.284047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:53:52.284047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:53:52.285252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:52.285252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:52.285252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:53:52.285252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:52.285252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:53:52.285252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:52.285252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:52.285252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:52.285252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:52.285252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:52.285252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:52.285252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:52.285252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:52.285252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:52.285252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:52.285252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:52.285252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:52.285252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:52.286203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:52.286203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:52.286203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:53:52.286203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:53:52.286203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:53:52.286203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:53:52.286203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:53:52.286203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:53:52.286203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 25 15:53:52.286203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:52.286203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 290/595] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:53:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:52.286203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7a3ce-3c2d-4b5a-bbd3-d9bf7f39fc80 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:52.292715 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 306/596] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:53:52 2022] PUT /identity/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/3622c5ccb7534e7c8e8342b4f6643bc3/roles/7f1996bcf7e54638b4f53b3d8ba83495 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:53:52.293448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32e40b-f664-46cf-961a-406fe27cb1ec None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:52.353877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:52.406727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7a3ce-3c2d-4b5a-bbd3-d9bf7f39fc80 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:52.416484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32e40b-f664-46cf-961a-406fe27cb1ec None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:52.456378 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b17368-d409-46fa-93cb-3dbffab1d620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:52.457080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b17368-d409-46fa-93cb-3dbffab1d620 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:52.457358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b17368-d409-46fa-93cb-3dbffab1d620 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:52.457628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b17368-d409-46fa-93cb-3dbffab1d620 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:52.458360 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b17368-d409-46fa-93cb-3dbffab1d620 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:52.458661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b17368-d409-46fa-93cb-3dbffab1d620 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:52.460035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b17368-d409-46fa-93cb-3dbffab1d620 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:52.460619 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030ea5c-ac6e-4a5d-a423-ea8d8b800d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:52.461796 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030ea5c-ac6e-4a5d-a423-ea8d8b800d8a None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:52.462463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030ea5c-ac6e-4a5d-a423-ea8d8b800d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:52.462906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030ea5c-ac6e-4a5d-a423-ea8d8b800d8a None admin] PATH_INFO: `/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/5cba086728824ec68fb0ee90fb3f99dd/roles/7f1996bcf7e54638b4f53b3d8ba83495` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:52.478299 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 291/597] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:52.478299 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9d4d9-72bf-483e-9b68-8bc21225bdde None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:52.478299 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9d4d9-72bf-483e-9b68-8bc21225bdde None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:52.478299 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9d4d9-72bf-483e-9b68-8bc21225bdde None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:52.480499 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030ea5c-ac6e-4a5d-a423-ea8d8b800d8a None admin] RBAC: Authorizing `identity:create_grant(project_id=3600648eb94b434b8c797d2f454e46b8, user_id=5cba086728824ec68fb0ee90fb3f99dd, role_id=7f1996bcf7e54638b4f53b3d8ba83495)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:52.480606 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 292/598] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:52.481626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9d4d9-72bf-483e-9b68-8bc21225bdde None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:52.481951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030ea5c-ac6e-4a5d-a423-ea8d8b800d8a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:52.496121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 307/599] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:53:52 2022] PUT /identity/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/5cba086728824ec68fb0ee90fb3f99dd/roles/7f1996bcf7e54638b4f53b3d8ba83495 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:53:52.497236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030ea5c-ac6e-4a5d-a423-ea8d8b800d8a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:52.548095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:52.613419 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9d4d9-72bf-483e-9b68-8bc21225bdde None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:52.618686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030ea5c-ac6e-4a5d-a423-ea8d8b800d8a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:52.630118 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:52.655747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17e3cf-85a5-49e8-bb04-e8a744af6e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:52.656489 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17e3cf-85a5-49e8-bb04-e8a744af6e25 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:52.656775 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17e3cf-85a5-49e8-bb04-e8a744af6e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:52.657050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17e3cf-85a5-49e8-bb04-e8a744af6e25 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:52.657914 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17e3cf-85a5-49e8-bb04-e8a744af6e25 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:52.658797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17e3cf-85a5-49e8-bb04-e8a744af6e25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:52.659698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17e3cf-85a5-49e8-bb04-e8a744af6e25 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:52.661015 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c1968-1c6f-4e75-bd2a-eca6df422583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:52.662000 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c1968-1c6f-4e75-bd2a-eca6df422583 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:52.662415 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c1968-1c6f-4e75-bd2a-eca6df422583 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:52.662820 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c1968-1c6f-4e75-bd2a-eca6df422583 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:52.671231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c1968-1c6f-4e75-bd2a-eca6df422583 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:52.672711 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c1968-1c6f-4e75-bd2a-eca6df422583 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:52.681361 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 293/600] 10.222.0.13 () {56 vars in 1139 bytes} [Sat Jun 25 15:53:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:52.682986 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed45804-d540-4136-9698-5671a18e8aec None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:52.683451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed45804-d540-4136-9698-5671a18e8aec None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:52.683901 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed45804-d540-4136-9698-5671a18e8aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:52.687438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-ba8c1968-1c6f-4e75-bd2a-eca6df422583 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:52.687438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:52.687438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:52.687438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:52.687438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:52.687438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:52.687438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:52.687438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:52.687438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:52.687438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:52.687438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:52.687438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:52.687438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:53:52.687438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:53:52.687438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:53:52.688555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:53:52.688555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:52.688555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:52.688555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:52.688555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:52.688555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:52.688555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:52.688555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:52.688555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:52.688555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:52.688555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:52.688555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:52.688555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:52.688555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:52.688555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:52.688555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:52.688555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:52.689646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:53:52.689646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:53:52.689646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:53:52.689646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:53:52.689646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:53:52.689646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:53:52.689646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:52.689646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:52.689646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 308/601] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:52.690257 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c1968-1c6f-4e75-bd2a-eca6df422583 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:52.707835 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-9ed45804-d540-4136-9698-5671a18e8aec None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:52.709791 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9ed45804-d540-4136-9698-5671a18e8aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:52.796522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 294/602] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:52.797457 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed45804-d540-4136-9698-5671a18e8aec None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:52.810591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:52.817525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c1968-1c6f-4e75-bd2a-eca6df422583 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:52.866706 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6980a88-ac77-4cd0-b4fd-aa23b0da5504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:52.867528 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6980a88-ac77-4cd0-b4fd-aa23b0da5504 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:52.868139 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6980a88-ac77-4cd0-b4fd-aa23b0da5504 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:52.868447 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6980a88-ac77-4cd0-b4fd-aa23b0da5504 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:52.880171 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6980a88-ac77-4cd0-b4fd-aa23b0da5504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:52.881732 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6980a88-ac77-4cd0-b4fd-aa23b0da5504 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:52.900648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-c6980a88-ac77-4cd0-b4fd-aa23b0da5504 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:52.900648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:52.900648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:52.900648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:52.900648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:52.900648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:52.900648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:52.900648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:52.900648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:52.900648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:52.900648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:52.900648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:52.900648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:52.900648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:52.900648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:52.901798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:52.901798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:52.901798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:52.901798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:52.901798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:52.901798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:52.901798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:52.901798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:52.901798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:52.901798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:52.901798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:52.901798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:52.901798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:52.901798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:52.901798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:52.901798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:52.901798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:52.902810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:52.902810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:52.902810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:52.902810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:52.902810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 309/603] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:52.902810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6980a88-ac77-4cd0-b4fd-aa23b0da5504 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:52.926498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed45804-d540-4136-9698-5671a18e8aec None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:52.928261 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:52.964786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa215c0c-0c45-4cd5-82e0-852306d376ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:52.965746 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa215c0c-0c45-4cd5-82e0-852306d376ce None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:52.966259 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa215c0c-0c45-4cd5-82e0-852306d376ce None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:52.966706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa215c0c-0c45-4cd5-82e0-852306d376ce None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:52.967696 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa215c0c-0c45-4cd5-82e0-852306d376ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:52.968144 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa215c0c-0c45-4cd5-82e0-852306d376ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:52.969934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa215c0c-0c45-4cd5-82e0-852306d376ce None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:52.987858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 295/604] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:53:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:52.989024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa215c0c-0c45-4cd5-82e0-852306d376ce None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:53.013394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6980a88-ac77-4cd0-b4fd-aa23b0da5504 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:53.057405 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08e467-b6fb-4903-b544-2cfa7629382b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:53.059103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08e467-b6fb-4903-b544-2cfa7629382b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:53.059582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08e467-b6fb-4903-b544-2cfa7629382b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:53.060050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08e467-b6fb-4903-b544-2cfa7629382b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:53.073774 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08e467-b6fb-4903-b544-2cfa7629382b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:53.083485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-8c08e467-b6fb-4903-b544-2cfa7629382b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:53.083485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:53.083485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:53.083485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:53.083485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:53.083485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:53.083485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:53.083485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:53.083485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:53.083485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:53.083485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:53.083485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:53.083485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:53.083485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:53.083485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:53.083485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:53.084640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:53.084640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:53.084640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:53.084640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:53.084640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:53.084640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:53.084640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:53.084640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:53.084640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:53.084640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:53.084640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:53.084640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:53.084640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:53.084640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:53.084640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:53.084640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:53.085703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:53.085703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:53.085703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:53.085703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:53.085703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:53.085703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:53.085703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:53.085703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:53.085703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:53.085703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:53.085703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:53.085703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:53.088991 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 310/605] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:53:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:53.089970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08e467-b6fb-4903-b544-2cfa7629382b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:53.104742 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:53.118243 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa215c0c-0c45-4cd5-82e0-852306d376ce None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:53.164369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087affcc-802b-4824-8afc-f235199bd9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:53.165222 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087affcc-802b-4824-8afc-f235199bd9fb None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:53.165812 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087affcc-802b-4824-8afc-f235199bd9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:53.166158 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087affcc-802b-4824-8afc-f235199bd9fb None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:53.167396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087affcc-802b-4824-8afc-f235199bd9fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:53.167762 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087affcc-802b-4824-8afc-f235199bd9fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:53.169775 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087affcc-802b-4824-8afc-f235199bd9fb None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:53.192019 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 296/606] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:53.192019 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087affcc-802b-4824-8afc-f235199bd9fb None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:53.223186 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:53.223426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08e467-b6fb-4903-b544-2cfa7629382b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:53.267146 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f20cc-479b-42e6-84f2-fe77710b0642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:53.268483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f20cc-479b-42e6-84f2-fe77710b0642 None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:53.268908 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f20cc-479b-42e6-84f2-fe77710b0642 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:53.269612 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f20cc-479b-42e6-84f2-fe77710b0642 None admin] PATH_INFO: `/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/b00152c72a1f4755b09c1dbd1d874b11/roles/7f1996bcf7e54638b4f53b3d8ba83495` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:53.288773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f20cc-479b-42e6-84f2-fe77710b0642 None admin] RBAC: Authorizing `identity:create_grant(project_id=3600648eb94b434b8c797d2f454e46b8, user_id=b00152c72a1f4755b09c1dbd1d874b11, role_id=7f1996bcf7e54638b4f53b3d8ba83495)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:53.290552 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f20cc-479b-42e6-84f2-fe77710b0642 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:53.308129 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 311/607] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:53:53 2022] PUT /identity/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/b00152c72a1f4755b09c1dbd1d874b11/roles/7f1996bcf7e54638b4f53b3d8ba83495 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:53:53.314671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f20cc-479b-42e6-84f2-fe77710b0642 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:53.339964 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087affcc-802b-4824-8afc-f235199bd9fb None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:53.415869 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db2b9b-8ed9-4785-b27e-cb5d5a26db97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:53.416843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db2b9b-8ed9-4785-b27e-cb5d5a26db97 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:53.417135 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db2b9b-8ed9-4785-b27e-cb5d5a26db97 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:53.417434 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db2b9b-8ed9-4785-b27e-cb5d5a26db97 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:53.418863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db2b9b-8ed9-4785-b27e-cb5d5a26db97 None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:53.419307 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db2b9b-8ed9-4785-b27e-cb5d5a26db97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:53.421460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db2b9b-8ed9-4785-b27e-cb5d5a26db97 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:53.436123 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 297/608] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:53:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:53.443012 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:53.443313 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db2b9b-8ed9-4785-b27e-cb5d5a26db97 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:53.519657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f20cc-479b-42e6-84f2-fe77710b0642 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:53.571126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e27b1-c957-493c-8935-33b804ef8720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:53.572586 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e27b1-c957-493c-8935-33b804ef8720 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:53.572870 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e27b1-c957-493c-8935-33b804ef8720 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:53.573144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e27b1-c957-493c-8935-33b804ef8720 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:53.575051 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:53.582473 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e27b1-c957-493c-8935-33b804ef8720 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:53.584106 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e27b1-c957-493c-8935-33b804ef8720 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:53.594400 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db2b9b-8ed9-4785-b27e-cb5d5a26db97 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:53.604120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-c41e27b1-c957-493c-8935-33b804ef8720 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:53.604120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:53.604120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:53.604120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:53.604120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:53.604120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:53.604120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:53.604120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:53.604120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:53.604120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:53.604120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:53.604120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:53.604120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:53:53.604120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:53:53.604120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:53:53.605140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:53:53.605140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:53.605140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:53.605140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:53.605140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:53.605140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:53.605140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:53.605140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:53.605140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:53.605140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:53.605140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:53.605140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:53.605140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:53.605140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:53.605140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:53.605140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:53.605140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:53.606033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:53:53.606033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:53:53.606033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:53:53.606033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:53:53.606033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:53:53.606033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:53:53.606033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:53.606033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:53.606470 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 312/609] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:53.612875 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e27b1-c957-493c-8935-33b804ef8720 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:53.632559 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad26233-c3b1-4fc4-b8d1-3dbaa3688ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:53.633615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad26233-c3b1-4fc4-b8d1-3dbaa3688ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:53.634009 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad26233-c3b1-4fc4-b8d1-3dbaa3688ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:53.634353 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad26233-c3b1-4fc4-b8d1-3dbaa3688ea9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:53.645836 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad26233-c3b1-4fc4-b8d1-3dbaa3688ea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:53.647307 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad26233-c3b1-4fc4-b8d1-3dbaa3688ea9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:53.665223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-7ad26233-c3b1-4fc4-b8d1-3dbaa3688ea9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:53.665223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:53.665223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:53.665223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:53.665223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:53.665223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:53.665223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:53.665223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:53.665223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:53.665223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:53.665223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:53.665223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:53.665223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:53.665223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:53.665223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:53.666244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:53.666244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:53.666244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:53.666244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:53.666244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:53.666244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:53.666244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:53.666244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:53.666244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:53.666244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:53.666244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:53.666244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:53.666244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:53.666244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:53.666244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:53.666244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:53.666244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:53.667356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:53.667356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:53.667356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:53.667356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:53.667356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 298/610] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:53.672165 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad26233-c3b1-4fc4-b8d1-3dbaa3688ea9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:53.780206 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad26233-c3b1-4fc4-b8d1-3dbaa3688ea9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:53.783661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e27b1-c957-493c-8935-33b804ef8720 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:53.812394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:53.817618 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad70e4-b0f8-4b7c-9217-6e4e4de2a2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:53.818295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad70e4-b0f8-4b7c-9217-6e4e4de2a2da None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:53.818578 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad70e4-b0f8-4b7c-9217-6e4e4de2a2da None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:53.818853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad70e4-b0f8-4b7c-9217-6e4e4de2a2da None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:53.819582 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad70e4-b0f8-4b7c-9217-6e4e4de2a2da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:53.819885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad70e4-b0f8-4b7c-9217-6e4e4de2a2da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:53.821355 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad70e4-b0f8-4b7c-9217-6e4e4de2a2da None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:53.824201 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:53.835034 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 299/611] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:53.840656 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad70e4-b0f8-4b7c-9217-6e4e4de2a2da None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:53.854684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7fbe49-fe23-474d-a7cd-09bf39516df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:53.855378 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fbe49-fe23-474d-a7cd-09bf39516df2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:53.855659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fbe49-fe23-474d-a7cd-09bf39516df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:53.860782 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7fbe49-fe23-474d-a7cd-09bf39516df2 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:53.860782 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7fbe49-fe23-474d-a7cd-09bf39516df2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:53.860782 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7fbe49-fe23-474d-a7cd-09bf39516df2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:53.860782 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7fbe49-fe23-474d-a7cd-09bf39516df2 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:53.886581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 313/612] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:53:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:53.892898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7fbe49-fe23-474d-a7cd-09bf39516df2 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:53.947053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad70e4-b0f8-4b7c-9217-6e4e4de2a2da None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:53.982725 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712cd88-4064-4c82-8c32-17fa0700d29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:53.983523 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712cd88-4064-4c82-8c32-17fa0700d29d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:53.983804 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712cd88-4064-4c82-8c32-17fa0700d29d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:53.984077 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712cd88-4064-4c82-8c32-17fa0700d29d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:53.991635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712cd88-4064-4c82-8c32-17fa0700d29d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:53.993257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712cd88-4064-4c82-8c32-17fa0700d29d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:54.008488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-5712cd88-4064-4c82-8c32-17fa0700d29d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 25 15:53:54.008488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:54.008488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:54.008488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:54.008488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:54.008488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:54.008488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:54.008488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:54.008488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:54.008488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:54.008488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:54.008488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:54.008488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:53:54.008488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:53:54.008488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:53:54.008488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:53:54.010340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:54.010340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:54.010340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:53:54.010340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:54.010340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:53:54.010340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:54.010340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:54.010340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:54.010340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:54.010340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:54.010340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:54.010340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:54.010340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:54.010340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:54.010340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:54.010340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:54.010340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:54.010340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:54.012242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:54.012242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:54.012242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:53:54.012242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:53:54.012242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:53:54.012242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:53:54.012242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:53:54.012242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:53:54.012242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 25 15:53:54.012242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:54.012242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 300/613] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:53:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:54.017901 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712cd88-4064-4c82-8c32-17fa0700d29d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:54.087191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7fbe49-fe23-474d-a7cd-09bf39516df2 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:54.136751 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:54.144947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a03f3e-e0f2-42ad-92e6-6cf17bb268e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:54.145933 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a03f3e-e0f2-42ad-92e6-6cf17bb268e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:54.146215 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a03f3e-e0f2-42ad-92e6-6cf17bb268e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:54.146498 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a03f3e-e0f2-42ad-92e6-6cf17bb268e8 None admin] PATH_INFO: `/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/aaffb9f0bf714822829455c0487376ca/roles/7f1996bcf7e54638b4f53b3d8ba83495` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:54.179155 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a03f3e-e0f2-42ad-92e6-6cf17bb268e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=3600648eb94b434b8c797d2f454e46b8, user_id=aaffb9f0bf714822829455c0487376ca, role_id=7f1996bcf7e54638b4f53b3d8ba83495)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:54.180745 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a03f3e-e0f2-42ad-92e6-6cf17bb268e8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:54.204669 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 314/614] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:53:53 2022] PUT /identity/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/aaffb9f0bf714822829455c0487376ca/roles/7f1996bcf7e54638b4f53b3d8ba83495 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 15:53:54.224721 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712cd88-4064-4c82-8c32-17fa0700d29d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:54.282547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129df6e-16dc-4a22-9b9c-8e490518db24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:54.283589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129df6e-16dc-4a22-9b9c-8e490518db24 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:54.284044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129df6e-16dc-4a22-9b9c-8e490518db24 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:54.284511 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129df6e-16dc-4a22-9b9c-8e490518db24 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:54.285546 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7129df6e-16dc-4a22-9b9c-8e490518db24 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:54.286029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7129df6e-16dc-4a22-9b9c-8e490518db24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:54.287721 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7129df6e-16dc-4a22-9b9c-8e490518db24 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:54.313561 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 301/615] 10.222.0.13 () {56 vars in 1135 bytes} [Sat Jun 25 15:53:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:54.319035 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a03f3e-e0f2-42ad-92e6-6cf17bb268e8 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:54.384523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:54.486361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a03f3e-e0f2-42ad-92e6-6cf17bb268e8 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:54.521400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8475596-4d1c-45ac-a04d-0e94bb6c775c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:54.522216 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8475596-4d1c-45ac-a04d-0e94bb6c775c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:54.522501 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8475596-4d1c-45ac-a04d-0e94bb6c775c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:54.522776 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8475596-4d1c-45ac-a04d-0e94bb6c775c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:54.528575 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e7f5f-ce28-4fe3-b2a0-d7a76aea0bf4 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:54.529140 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e7f5f-ce28-4fe3-b2a0-d7a76aea0bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:54.529605 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e7f5f-ce28-4fe3-b2a0-d7a76aea0bf4 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:54.531132 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 302/616] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:53:54.536244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10f264-a983-48a2-8595-014b7b5785d0 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:54.537187 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10f264-a983-48a2-8595-014b7b5785d0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:54.537676 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10f264-a983-48a2-8595-014b7b5785d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:54.543402 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8475596-4d1c-45ac-a04d-0e94bb6c775c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:54.543402 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8475596-4d1c-45ac-a04d-0e94bb6c775c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:54.563886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-c8475596-4d1c-45ac-a04d-0e94bb6c775c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:54.563886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:54.563886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:54.563886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:54.563886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:54.563886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:54.563886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:54.563886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:54.563886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:54.563886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:54.563886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:54.563886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:54.563886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:54.563886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:54.563886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:54.564880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:54.564880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:54.564880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:54.564880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:54.564880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:54.564880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:54.564880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:54.564880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:54.564880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:54.564880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:54.564880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:54.564880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:54.564880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:54.564880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:54.564880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:54.564880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:54.564880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:54.565749 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:54.565749 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:54.565749 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:54.565749 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:54.565749 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 315/617] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:54.572441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8475596-4d1c-45ac-a04d-0e94bb6c775c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:54.584307 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2e10f264-a983-48a2-8595-014b7b5785d0 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:54.585827 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2e10f264-a983-48a2-8595-014b7b5785d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:54.763090 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 303/618] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:54.773895 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:54.798595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10f264-a983-48a2-8595-014b7b5785d0 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:54.847598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8475596-4d1c-45ac-a04d-0e94bb6c775c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:54.889733 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:54.926030 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c7a00-c92e-4b6b-be7a-4edd57925c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:54.927747 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c7a00-c92e-4b6b-be7a-4edd57925c1d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:54.929082 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c7a00-c92e-4b6b-be7a-4edd57925c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:54.929812 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c7a00-c92e-4b6b-be7a-4edd57925c1d None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:54.931975 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c7a00-c92e-4b6b-be7a-4edd57925c1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:54.933663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c7a00-c92e-4b6b-be7a-4edd57925c1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:54.937103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c7a00-c92e-4b6b-be7a-4edd57925c1d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:54.959179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 316/619] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:54.961036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380f985-36f8-4e7d-8b72-d1bf5521575a None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:54.961483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380f985-36f8-4e7d-8b72-d1bf5521575a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:54.962115 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380f985-36f8-4e7d-8b72-d1bf5521575a None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:54.966072 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 317/620] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:54.970838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380f985-36f8-4e7d-8b72-d1bf5521575a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:54.980817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10f264-a983-48a2-8595-014b7b5785d0 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:55.021159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac370f-423b-4aa6-8fc3-9254e4e79b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:55.021893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac370f-423b-4aa6-8fc3-9254e4e79b59 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:55.022175 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac370f-423b-4aa6-8fc3-9254e4e79b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:55.022450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac370f-423b-4aa6-8fc3-9254e4e79b59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:55.040165 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac370f-423b-4aa6-8fc3-9254e4e79b59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:55.052100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-e2ac370f-423b-4aa6-8fc3-9254e4e79b59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:55.052100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:55.052100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:55.052100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:55.052100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:55.052100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:55.052100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:55.052100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:55.052100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:55.052100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:55.052100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:55.052100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:55.052100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:55.052100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:55.052100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:55.052100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:55.053292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:55.053292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:55.053292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:55.053292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:55.053292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:55.053292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:55.053292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:55.053292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:55.053292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:55.053292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:55.053292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:55.053292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:55.053292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:55.053292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:55.053292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:55.053292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:55.054130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:55.054130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:55.054130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:55.054130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:55.054130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:55.054130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:55.054130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:55.054130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:55.054130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:55.054130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:55.054130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:55.054130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:55.054803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 304/621] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:53:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:55.058917 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55244516-e78c-44cb-b4f4-f032afee8cbb None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:55.058917 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55244516-e78c-44cb-b4f4-f032afee8cbb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:55.058917 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55244516-e78c-44cb-b4f4-f032afee8cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:55.083269 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-55244516-e78c-44cb-b4f4-f032afee8cbb None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:55.084526 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-55244516-e78c-44cb-b4f4-f032afee8cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:55.115266 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380f985-36f8-4e7d-8b72-d1bf5521575a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:55.151308 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b68f66-45d2-467b-b5e4-ebcee3b6cfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:55.152211 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b68f66-45d2-467b-b5e4-ebcee3b6cfaf None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:55.152585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b68f66-45d2-467b-b5e4-ebcee3b6cfaf None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:55.152927 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b68f66-45d2-467b-b5e4-ebcee3b6cfaf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:55.162106 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b68f66-45d2-467b-b5e4-ebcee3b6cfaf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:55.162305 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 305/622] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:55.163165 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55244516-e78c-44cb-b4f4-f032afee8cbb None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:55.163857 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b68f66-45d2-467b-b5e4-ebcee3b6cfaf None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:55.178773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-65b68f66-45d2-467b-b5e4-ebcee3b6cfaf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:55.178773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:55.178773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:55.178773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:55.178773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:55.178773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:55.178773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:55.178773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:55.178773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:55.178773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:55.178773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:55.178773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:55.178773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:53:55.178773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:53:55.178773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:53:55.180724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:53:55.180724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:55.180724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:55.180724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:55.180724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:55.180724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:55.180724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:55.180724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:55.180724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:55.180724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:55.180724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:55.180724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:55.180724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:55.180724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:55.180724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:55.180724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:55.180724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:55.182012 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:53:55.182012 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:53:55.182012 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:53:55.182012 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:53:55.182012 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:53:55.182012 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:53:55.182012 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:55.182012 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:55.182012 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 318/623] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:55.192438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b68f66-45d2-467b-b5e4-ebcee3b6cfaf None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:55.194010 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:55.199530 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:55.276049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55244516-e78c-44cb-b4f4-f032afee8cbb None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:55.297955 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b68f66-45d2-467b-b5e4-ebcee3b6cfaf None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:55.318805 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf4a8b-47ed-4f94-8d6a-f758adc67932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:55.319612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf4a8b-47ed-4f94-8d6a-f758adc67932 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:55.319996 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf4a8b-47ed-4f94-8d6a-f758adc67932 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:55.321705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf4a8b-47ed-4f94-8d6a-f758adc67932 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:55.325188 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf4a8b-47ed-4f94-8d6a-f758adc67932 None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:55.325500 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf4a8b-47ed-4f94-8d6a-f758adc67932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:55.326916 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf4a8b-47ed-4f94-8d6a-f758adc67932 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:55.339848 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 306/624] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:53:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:55.341078 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf4a8b-47ed-4f94-8d6a-f758adc67932 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:55.342913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3a9b6-5069-4776-b5eb-7f52ac26c8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:55.343652 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3a9b6-5069-4776-b5eb-7f52ac26c8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:55.344003 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3a9b6-5069-4776-b5eb-7f52ac26c8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:55.344483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3a9b6-5069-4776-b5eb-7f52ac26c8c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:55.351322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3a9b6-5069-4776-b5eb-7f52ac26c8c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:55.351741 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3a9b6-5069-4776-b5eb-7f52ac26c8c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:55.353341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3a9b6-5069-4776-b5eb-7f52ac26c8c5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:55.366367 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 319/625] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:53:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:55.367363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3a9b6-5069-4776-b5eb-7f52ac26c8c5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:55.430668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:55.443035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf4a8b-47ed-4f94-8d6a-f758adc67932 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:55.444743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:55.478151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3a9b6-5069-4776-b5eb-7f52ac26c8c5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:55.486640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b29fb6-8344-41dc-a2ce-0ef828e221f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:55.487437 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b29fb6-8344-41dc-a2ce-0ef828e221f3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:55.487710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b29fb6-8344-41dc-a2ce-0ef828e221f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:55.487974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b29fb6-8344-41dc-a2ce-0ef828e221f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:55.500225 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b29fb6-8344-41dc-a2ce-0ef828e221f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:55.510392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-94b29fb6-8344-41dc-a2ce-0ef828e221f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:55.510392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:55.510392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:55.510392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:55.510392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:55.510392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:55.510392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:55.510392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:55.510392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:55.510392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:55.510392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:55.510392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:55.510392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:55.510392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:55.510392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:55.510392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:55.511462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:55.511462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:55.511462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:55.511462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:55.511462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:55.511462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:55.511462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:55.511462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:55.511462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:55.511462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:55.511462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:55.511462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:55.511462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:55.511462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:55.511462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:55.511462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:55.512392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:55.512392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:55.512392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:55.512392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:55.512392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:55.512392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:55.512392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:55.512392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:55.512392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:55.512392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:55.512392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:55.512392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:55.516685 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 307/626] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:53:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:55.517908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b29fb6-8344-41dc-a2ce-0ef828e221f3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:55.528545 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75bbd8-b42d-4bb8-8589-e2afb6090ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:55.529637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75bbd8-b42d-4bb8-8589-e2afb6090ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:55.530150 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75bbd8-b42d-4bb8-8589-e2afb6090ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:55.530621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75bbd8-b42d-4bb8-8589-e2afb6090ce7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:55.541974 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75bbd8-b42d-4bb8-8589-e2afb6090ce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:55.543687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75bbd8-b42d-4bb8-8589-e2afb6090ce7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:55.562394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-6e75bbd8-b42d-4bb8-8589-e2afb6090ce7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:55.562394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:55.562394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:55.562394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:55.562394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:55.562394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:55.562394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:55.562394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:55.562394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:55.562394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:55.562394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:55.562394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:55.562394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:55.562394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:55.562394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:55.563347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:55.563347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:55.563347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:55.563347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:55.563347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:55.563347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:55.563347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:55.563347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:55.563347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:55.563347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:55.563347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:55.563347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:55.563347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:55.563347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:55.563347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:55.563347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:55.563347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:55.564272 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:55.564272 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:55.564272 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:55.564272 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:55.566683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 320/627] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:55.567670 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75bbd8-b42d-4bb8-8589-e2afb6090ce7 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:55.631418 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b29fb6-8344-41dc-a2ce-0ef828e221f3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:55.668098 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496f135-734f-4c78-bd7c-54f27c711660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:55.669117 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496f135-734f-4c78-bd7c-54f27c711660 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:55.669395 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496f135-734f-4c78-bd7c-54f27c711660 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:55.669672 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496f135-734f-4c78-bd7c-54f27c711660 None admin] PATH_INFO: `/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/19788c4239ca44f9a31a7d32086c8947/roles/7f1996bcf7e54638b4f53b3d8ba83495` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:55.676341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75bbd8-b42d-4bb8-8589-e2afb6090ce7 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:55.689160 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496f135-734f-4c78-bd7c-54f27c711660 None admin] RBAC: Authorizing `identity:create_grant(project_id=3600648eb94b434b8c797d2f454e46b8, user_id=19788c4239ca44f9a31a7d32086c8947, role_id=7f1996bcf7e54638b4f53b3d8ba83495)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:55.690974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496f135-734f-4c78-bd7c-54f27c711660 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:55.691345 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:55.711014 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 308/628] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:53:55 2022] PUT /identity/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/19788c4239ca44f9a31a7d32086c8947/roles/7f1996bcf7e54638b4f53b3d8ba83495 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:53:55.712308 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496f135-734f-4c78-bd7c-54f27c711660 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:55.722689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:55.731105 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625bd63-a39c-4162-92ff-726f81027ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:55.731792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625bd63-a39c-4162-92ff-726f81027ddd None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:55.732073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625bd63-a39c-4162-92ff-726f81027ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:55.732333 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625bd63-a39c-4162-92ff-726f81027ddd None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:55.733145 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625bd63-a39c-4162-92ff-726f81027ddd None admin] RBAC: Adding query filter params (name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:55.733433 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625bd63-a39c-4162-92ff-726f81027ddd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:55.734847 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625bd63-a39c-4162-92ff-726f81027ddd None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:55.751890 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 321/629] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:53:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:55.757742 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625bd63-a39c-4162-92ff-726f81027ddd None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:55.905446 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496f135-734f-4c78-bd7c-54f27c711660 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:55.932032 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625bd63-a39c-4162-92ff-726f81027ddd None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:55.958324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ee086-6a8a-494b-9b6d-6b656d32d64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:55.959748 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ee086-6a8a-494b-9b6d-6b656d32d64e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:55.960204 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ee086-6a8a-494b-9b6d-6b656d32d64e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:55.960566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ee086-6a8a-494b-9b6d-6b656d32d64e None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:55.962176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ee086-6a8a-494b-9b6d-6b656d32d64e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:55.962496 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ee086-6a8a-494b-9b6d-6b656d32d64e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:55.964382 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ee086-6a8a-494b-9b6d-6b656d32d64e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:55.970676 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619718f3-aede-4d90-8eb4-3659b61f2df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:55.971478 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619718f3-aede-4d90-8eb4-3659b61f2df8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:55.971759 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619718f3-aede-4d90-8eb4-3659b61f2df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:55.972029 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619718f3-aede-4d90-8eb4-3659b61f2df8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:55.979901 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 309/630] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:55.985070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ee086-6a8a-494b-9b6d-6b656d32d64e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:55.986507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619718f3-aede-4d90-8eb4-3659b61f2df8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:55.987982 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619718f3-aede-4d90-8eb4-3659b61f2df8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:56.007092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-619718f3-aede-4d90-8eb4-3659b61f2df8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:56.007092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:56.007092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:56.007092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:56.007092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:56.007092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:56.007092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:56.007092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:56.007092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:56.007092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:56.007092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:56.007092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:56.007092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:56.007092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:56.007092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:56.008038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:56.008038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:56.008038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:56.008038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:56.008038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:56.008038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:56.008038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:56.008038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:56.008038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:56.008038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:56.008038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:56.008038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:56.008038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:56.008038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:56.008038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:56.008038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:56.008038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:56.009683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:56.009683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:56.009683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:56.009683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:56.009683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 322/631] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:53:56.017704 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619718f3-aede-4d90-8eb4-3659b61f2df8 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:56.047357 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:56.121159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:56.122384 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619718f3-aede-4d90-8eb4-3659b61f2df8 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:56.157468 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee788c-415a-40dc-8b1d-5c7188f1821d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:56.158128 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee788c-415a-40dc-8b1d-5c7188f1821d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:56.158405 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee788c-415a-40dc-8b1d-5c7188f1821d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:56.158678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee788c-415a-40dc-8b1d-5c7188f1821d None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:56.159398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee788c-415a-40dc-8b1d-5c7188f1821d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:56.159702 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee788c-415a-40dc-8b1d-5c7188f1821d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:56.161083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee788c-415a-40dc-8b1d-5c7188f1821d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:56.164087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ee086-6a8a-494b-9b6d-6b656d32d64e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:56.178062 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 323/632] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:56.179466 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70659be1-48ac-415f-bc76-d3e6d592a712 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:56.179753 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70659be1-48ac-415f-bc76-d3e6d592a712 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:56.180027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70659be1-48ac-415f-bc76-d3e6d592a712 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:56.185573 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 324/633] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:56.186364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70659be1-48ac-415f-bc76-d3e6d592a712 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:56.212866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd7172-0b0a-4cf4-9459-90b720d955bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:56.213737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd7172-0b0a-4cf4-9459-90b720d955bd None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:56.214562 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd7172-0b0a-4cf4-9459-90b720d955bd None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:56.214841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd7172-0b0a-4cf4-9459-90b720d955bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:56.225897 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd7172-0b0a-4cf4-9459-90b720d955bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:56.227349 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd7172-0b0a-4cf4-9459-90b720d955bd None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:56.245714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-c3fd7172-0b0a-4cf4-9459-90b720d955bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 25 15:53:56.245714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:56.245714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:56.245714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:56.245714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:56.245714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:56.245714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:56.245714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:56.245714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:56.245714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:56.245714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:56.245714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:56.245714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:53:56.245714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:53:56.245714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:53:56.245714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:53:56.246767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:56.246767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:56.246767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:53:56.246767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:56.246767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:53:56.246767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:56.246767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:56.246767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:56.246767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:56.246767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:56.246767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:56.246767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:56.246767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:56.246767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:56.246767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:56.246767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:56.246767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:56.246767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:56.247719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:56.247719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:56.247719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:53:56.247719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:53:56.247719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:53:56.247719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:53:56.247719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:53:56.247719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:53:56.247719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 25 15:53:56.247719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:56.251790 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 310/634] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Jun 25 15:53:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:56.253192 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e4ffe-9ba9-41a5-9480-fbb74f7cf3b0 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:56.253473 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e4ffe-9ba9-41a5-9480-fbb74f7cf3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:56.253743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e4ffe-9ba9-41a5-9480-fbb74f7cf3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:56.276913 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-fe1e4ffe-9ba9-41a5-9480-fbb74f7cf3b0 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:56.278107 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-fe1e4ffe-9ba9-41a5-9480-fbb74f7cf3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:56.294413 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:56.308912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70659be1-48ac-415f-bc76-d3e6d592a712 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:56.345870 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a52046-915f-4c9b-af83-4662867f466d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:56.346692 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a52046-915f-4c9b-af83-4662867f466d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:56.347035 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a52046-915f-4c9b-af83-4662867f466d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:56.347373 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a52046-915f-4c9b-af83-4662867f466d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:56.354636 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 311/635] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:56.355805 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e4ffe-9ba9-41a5-9480-fbb74f7cf3b0 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:56.361637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a52046-915f-4c9b-af83-4662867f466d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:56.363100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a52046-915f-4c9b-af83-4662867f466d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:56.391059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-26a52046-915f-4c9b-af83-4662867f466d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 25 15:53:56.391059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:56.391059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:56.391059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:56.391059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:56.391059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:56.391059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:56.391059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:56.391059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:56.391059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:56.391059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:56.391059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:56.391059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:53:56.391059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:53:56.391059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:53:56.391059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:53:56.392590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:56.392590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:56.392590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:53:56.392590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:56.392590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:53:56.392590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:56.392590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:56.392590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:56.392590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:56.392590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:56.392590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:56.392590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:56.392590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:56.392590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:56.392590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:56.392590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:56.392590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:56.392590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:56.393644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:56.393644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:56.393644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:53:56.393644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:53:56.393644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:53:56.393644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:53:56.393644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:53:56.393644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:53:56.393644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 25 15:53:56.393644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:56.393644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 325/636] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:56.398376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a52046-915f-4c9b-af83-4662867f466d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:56.436876 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:56.464183 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e4ffe-9ba9-41a5-9480-fbb74f7cf3b0 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:56.500610 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ca4ba-dcb4-47ea-8c89-cea2e48b9f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:56.501611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ca4ba-dcb4-47ea-8c89-cea2e48b9f8b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:56.502030 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ca4ba-dcb4-47ea-8c89-cea2e48b9f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:56.502250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a52046-915f-4c9b-af83-4662867f466d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:56.502607 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ca4ba-dcb4-47ea-8c89-cea2e48b9f8b None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:56.503673 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ca4ba-dcb4-47ea-8c89-cea2e48b9f8b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:56.504115 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ca4ba-dcb4-47ea-8c89-cea2e48b9f8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:56.505809 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ca4ba-dcb4-47ea-8c89-cea2e48b9f8b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:56.529775 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 312/637] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Jun 25 15:53:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:56.530890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ca4ba-dcb4-47ea-8c89-cea2e48b9f8b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:56.537935 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2de4b0-6fc3-4e3c-9d0f-ad6a7478a34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:56.538613 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2de4b0-6fc3-4e3c-9d0f-ad6a7478a34d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:56.538892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2de4b0-6fc3-4e3c-9d0f-ad6a7478a34d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:56.539161 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2de4b0-6fc3-4e3c-9d0f-ad6a7478a34d None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:56.540304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2de4b0-6fc3-4e3c-9d0f-ad6a7478a34d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:56.540685 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2de4b0-6fc3-4e3c-9d0f-ad6a7478a34d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:56.542120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2de4b0-6fc3-4e3c-9d0f-ad6a7478a34d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:56.565167 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 326/638] 10.222.0.13 () {56 vars in 1137 bytes} [Sat Jun 25 15:53:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:53:56.566040 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2de4b0-6fc3-4e3c-9d0f-ad6a7478a34d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:56.574381 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:56.633789 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ca4ba-dcb4-47ea-8c89-cea2e48b9f8b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:56.648300 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:56.669228 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2de4b0-6fc3-4e3c-9d0f-ad6a7478a34d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:56.670635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e70f2ce-86c5-4212-a786-a34bc236b568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:56.671482 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70f2ce-86c5-4212-a786-a34bc236b568 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:56.671761 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70f2ce-86c5-4212-a786-a34bc236b568 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:56.672027 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70f2ce-86c5-4212-a786-a34bc236b568 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:56.682090 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e70f2ce-86c5-4212-a786-a34bc236b568 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:56.689679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-6e70f2ce-86c5-4212-a786-a34bc236b568 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:56.689679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:56.689679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:56.689679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:56.689679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:56.689679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:56.689679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:56.689679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:56.689679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:56.689679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:56.689679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:56.689679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:56.689679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:56.689679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:56.689679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:56.689679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:56.690601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:56.690601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:56.690601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:56.690601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:56.690601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:56.690601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:56.690601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:56.690601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:56.690601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:56.690601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:56.690601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:56.690601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:56.690601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:56.690601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:56.690601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:56.690601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:56.691445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:56.691445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:56.691445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:56.691445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:56.691445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:56.691445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:56.691445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:56.691445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:56.691445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:56.691445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:56.691445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:56.691445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:56.691445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 313/639] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:53:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:56.692130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79592a7-422f-4202-861e-4053dc72a2a2 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:56.692813 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79592a7-422f-4202-861e-4053dc72a2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:56.692813 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79592a7-422f-4202-861e-4053dc72a2a2 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:56.696527 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 314/640] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:56.696527 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79592a7-422f-4202-861e-4053dc72a2a2 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:56.708839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7641a-25af-4760-a60d-3dbc1818a765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:56.709492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7641a-25af-4760-a60d-3dbc1818a765 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:56.709773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7641a-25af-4760-a60d-3dbc1818a765 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:56.710046 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7641a-25af-4760-a60d-3dbc1818a765 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:56.777408 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7641a-25af-4760-a60d-3dbc1818a765 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:56.778914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7641a-25af-4760-a60d-3dbc1818a765 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:56.787021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:56.806663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79592a7-422f-4202-861e-4053dc72a2a2 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:56.845257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea65e0-4124-4dd0-8d0e-f98a0356bf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:56.845910 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea65e0-4124-4dd0-8d0e-f98a0356bf4f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:56.846187 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea65e0-4124-4dd0-8d0e-f98a0356bf4f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:56.846456 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea65e0-4124-4dd0-8d0e-f98a0356bf4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:56.888356 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 327/641] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:56.889077 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7641a-25af-4760-a60d-3dbc1818a765 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:56.896122 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:56.922359 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea65e0-4124-4dd0-8d0e-f98a0356bf4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:56.923781 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea65e0-4124-4dd0-8d0e-f98a0356bf4f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:56.998208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7641a-25af-4760-a60d-3dbc1818a765 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:57.005595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:57.032012 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 315/642] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:57.033404 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca3cd5-8d28-4d7a-aeec-a18e961a123a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:57.033697 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca3cd5-8d28-4d7a-aeec-a18e961a123a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:57.033971 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca3cd5-8d28-4d7a-aeec-a18e961a123a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:57.038915 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1b9e9-ebf6-46be-8bcb-1cfdb0805cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:57.039723 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1b9e9-ebf6-46be-8bcb-1cfdb0805cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:57.040068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1b9e9-ebf6-46be-8bcb-1cfdb0805cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:57.040423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1b9e9-ebf6-46be-8bcb-1cfdb0805cf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:57.041223 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1b9e9-ebf6-46be-8bcb-1cfdb0805cf4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:57.041522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1b9e9-ebf6-46be-8bcb-1cfdb0805cf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:57.042883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1b9e9-ebf6-46be-8bcb-1cfdb0805cf4 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:57.051375 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 328/643] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:53:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:57.052187 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1b9e9-ebf6-46be-8bcb-1cfdb0805cf4 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:57.053230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-cfca3cd5-8d28-4d7a-aeec-a18e961a123a None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:57.054594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-cfca3cd5-8d28-4d7a-aeec-a18e961a123a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:57.126425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 316/644] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:57.127637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca3cd5-8d28-4d7a-aeec-a18e961a123a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:57.137752 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:57.160918 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1b9e9-ebf6-46be-8bcb-1cfdb0805cf4 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:57.198045 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4bdd2-8845-479d-9b83-35fdecfe6239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:57.198846 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4bdd2-8845-479d-9b83-35fdecfe6239 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:57.199132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4bdd2-8845-479d-9b83-35fdecfe6239 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:57.199402 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4bdd2-8845-479d-9b83-35fdecfe6239 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:57.210483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4bdd2-8845-479d-9b83-35fdecfe6239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:57.212034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4bdd2-8845-479d-9b83-35fdecfe6239 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:57.229418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-13f4bdd2-8845-479d-9b83-35fdecfe6239 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:57.229418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:57.229418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:57.229418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:57.229418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:57.229418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:57.229418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:57.229418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:57.229418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:57.229418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:57.229418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:57.229418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:57.229418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:57.229418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:57.229418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:57.230393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:57.230393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:57.230393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:57.230393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:57.230393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:57.230393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:57.230393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:57.230393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:57.230393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:57.230393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:57.230393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:57.230393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:57.230393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:57.230393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:57.230393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:57.230393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:57.230393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:57.231286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:57.231286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:57.231286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:57.231286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:57.231286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 329/645] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:57.231571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4bdd2-8845-479d-9b83-35fdecfe6239 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:57.248257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca3cd5-8d28-4d7a-aeec-a18e961a123a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:57.255078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:57.286910 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598b71d-fc5f-421f-ac3c-c57ac07cdd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:57.287735 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598b71d-fc5f-421f-ac3c-c57ac07cdd98 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:57.288015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598b71d-fc5f-421f-ac3c-c57ac07cdd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:57.288293 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598b71d-fc5f-421f-ac3c-c57ac07cdd98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:57.298796 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598b71d-fc5f-421f-ac3c-c57ac07cdd98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:57.300247 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598b71d-fc5f-421f-ac3c-c57ac07cdd98 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:57.317612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-7598b71d-fc5f-421f-ac3c-c57ac07cdd98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:57.317612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:57.317612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:57.317612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:57.317612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:57.317612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:57.317612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:57.317612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:57.317612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:57.317612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:57.317612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:57.317612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:57.317612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:57.317612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:57.317612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:57.318524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:57.318524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:57.318524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:57.318524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:57.318524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:57.318524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:57.318524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:57.318524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:57.318524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:57.318524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:57.318524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:57.318524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:57.318524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:57.318524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:57.318524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:57.318524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:57.318524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:57.320102 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:57.320102 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:57.320102 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:57.320102 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:57.320102 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 317/646] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:57.320102 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598b71d-fc5f-421f-ac3c-c57ac07cdd98 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:57.343328 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4bdd2-8845-479d-9b83-35fdecfe6239 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:57.380175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a27027-89c6-4caa-ae0d-9e0fe964a193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:57.381038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a27027-89c6-4caa-ae0d-9e0fe964a193 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:57.381322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a27027-89c6-4caa-ae0d-9e0fe964a193 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:57.381591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a27027-89c6-4caa-ae0d-9e0fe964a193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:57.393122 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a27027-89c6-4caa-ae0d-9e0fe964a193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:57.394571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a27027-89c6-4caa-ae0d-9e0fe964a193 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:57.399422 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:57.412785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-89a27027-89c6-4caa-ae0d-9e0fe964a193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:57.412785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:57.412785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:57.412785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:57.412785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:57.412785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:57.412785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:57.412785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:57.412785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:57.412785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:57.412785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:57.412785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:57.412785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:57.412785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:57.412785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:57.413722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:57.413722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:57.413722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:57.413722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:57.413722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:57.413722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:57.413722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:57.413722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:57.413722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:57.413722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:57.413722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:57.413722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:57.413722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:57.413722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:57.413722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:57.413722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:57.413722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:57.415316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:57.415316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:57.415316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:57.415316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:57.415316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 330/647] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:57.415316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a27027-89c6-4caa-ae0d-9e0fe964a193 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:57.449984 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598b71d-fc5f-421f-ac3c-c57ac07cdd98 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:57.485001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ae7f6-acb3-4276-ae88-65fd010727c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:57.485791 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ae7f6-acb3-4276-ae88-65fd010727c2 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:57.486058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ae7f6-acb3-4276-ae88-65fd010727c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:57.486323 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ae7f6-acb3-4276-ae88-65fd010727c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:57.496362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:57.496964 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ae7f6-acb3-4276-ae88-65fd010727c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:57.505005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-021ae7f6-acb3-4276-ae88-65fd010727c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:57.505005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:57.505005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:57.505005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:57.505005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:57.505005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:57.505005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:57.505005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:57.505005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:57.505005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:57.505005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:57.505005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:57.505005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:57.505005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:57.505005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:57.505005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:57.505956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:57.505956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:57.505956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:57.505956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:57.505956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:57.505956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:57.505956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:57.505956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:57.505956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:57.505956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:57.505956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:57.505956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:57.505956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:57.505956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:57.505956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:57.505956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:57.506807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:57.506807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:57.506807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:57.506807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:57.506807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:57.506807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:57.506807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:57.506807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:57.506807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:57.506807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:57.506807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:57.506807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:57.506807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 318/648] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:53:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:57.507508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ae7f6-acb3-4276-ae88-65fd010727c2 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:57.532559 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a27027-89c6-4caa-ae0d-9e0fe964a193 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:57.568394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a1289-d29a-4f61-96e2-8e9f7690b1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:57.569070 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a1289-d29a-4f61-96e2-8e9f7690b1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:57.569350 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a1289-d29a-4f61-96e2-8e9f7690b1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:57.569620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a1289-d29a-4f61-96e2-8e9f7690b1c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:57.570339 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a1289-d29a-4f61-96e2-8e9f7690b1c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:57.570643 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a1289-d29a-4f61-96e2-8e9f7690b1c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:57.572213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48a1289-d29a-4f61-96e2-8e9f7690b1c6 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:57.584666 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 331/649] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:57.589440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a1289-d29a-4f61-96e2-8e9f7690b1c6 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:57.616845 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ae7f6-acb3-4276-ae88-65fd010727c2 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:57.633101 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:57.652616 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9f5e4-ebbf-49a1-8441-f68bb6872c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:57.653284 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9f5e4-ebbf-49a1-8441-f68bb6872c03 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:57.653560 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9f5e4-ebbf-49a1-8441-f68bb6872c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:57.653828 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9f5e4-ebbf-49a1-8441-f68bb6872c03 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:57.654553 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e9f5e4-ebbf-49a1-8441-f68bb6872c03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:57.654856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e9f5e4-ebbf-49a1-8441-f68bb6872c03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:57.656232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e9f5e4-ebbf-49a1-8441-f68bb6872c03 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:57.669100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 319/650] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:57.669810 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9f5e4-ebbf-49a1-8441-f68bb6872c03 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:57.700463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a1289-d29a-4f61-96e2-8e9f7690b1c6 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:57.730002 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:57.736221 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0c366-9003-4f78-84f8-a58248d5e488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:57.737040 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0c366-9003-4f78-84f8-a58248d5e488 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:57.737426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0c366-9003-4f78-84f8-a58248d5e488 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:57.737765 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0c366-9003-4f78-84f8-a58248d5e488 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:57.738560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0c366-9003-4f78-84f8-a58248d5e488 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:57.738937 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0c366-9003-4f78-84f8-a58248d5e488 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:57.740377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0c366-9003-4f78-84f8-a58248d5e488 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:57.757320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 332/651] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:57.757320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0c366-9003-4f78-84f8-a58248d5e488 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:57.778389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9f5e4-ebbf-49a1-8441-f68bb6872c03 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:57.815650 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b9705-af28-49b9-a45c-14daf9449c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:57.816840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b9705-af28-49b9-a45c-14daf9449c84 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:57.817649 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b9705-af28-49b9-a45c-14daf9449c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:57.818068 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b9705-af28-49b9-a45c-14daf9449c84 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:57.818978 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b9705-af28-49b9-a45c-14daf9449c84 None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:57.819429 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b9705-af28-49b9-a45c-14daf9449c84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:57.821113 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b9705-af28-49b9-a45c-14daf9449c84 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:57.834034 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 320/652] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:53:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:57.834212 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b9705-af28-49b9-a45c-14daf9449c84 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:57.858496 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0c366-9003-4f78-84f8-a58248d5e488 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:57.875752 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:57.892555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63f95e-43b8-45a2-a174-64392f94e527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:57.893546 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63f95e-43b8-45a2-a174-64392f94e527 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:57.893984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63f95e-43b8-45a2-a174-64392f94e527 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:57.894261 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63f95e-43b8-45a2-a174-64392f94e527 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:57.901231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63f95e-43b8-45a2-a174-64392f94e527 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:57.902685 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63f95e-43b8-45a2-a174-64392f94e527 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:57.916484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-5a63f95e-43b8-45a2-a174-64392f94e527 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:57.916484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:57.916484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:57.916484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:57.916484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:57.916484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:57.916484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:57.916484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:57.916484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:57.916484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:57.916484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:57.916484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:57.916484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:53:57.916484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:53:57.916484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:53:57.917377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:53:57.917377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:57.917377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:57.917377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:57.917377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:57.917377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:57.917377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:57.917377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:57.917377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:57.917377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:57.917377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:57.917377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:57.917377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:57.917377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:57.917377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:57.917377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:57.917377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:57.918256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:53:57.918256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:53:57.918256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:53:57.918256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:53:57.918256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:53:57.918256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:53:57.918256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:57.918256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:57.918256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 333/653] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:57.918762 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63f95e-43b8-45a2-a174-64392f94e527 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:57.937151 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b9705-af28-49b9-a45c-14daf9449c84 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:57.969676 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:57.972747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60164b-9866-4687-93aa-0ff0ead15670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:57.973552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60164b-9866-4687-93aa-0ff0ead15670 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:57.973818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60164b-9866-4687-93aa-0ff0ead15670 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:57.974086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e60164b-9866-4687-93aa-0ff0ead15670 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:57.980913 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60164b-9866-4687-93aa-0ff0ead15670 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:57.982355 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e60164b-9866-4687-93aa-0ff0ead15670 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:57.996287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-3e60164b-9866-4687-93aa-0ff0ead15670 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:57.996287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:57.996287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:57.996287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:57.996287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:57.996287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:57.996287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:57.996287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:57.996287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:57.996287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:57.996287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:57.996287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:57.996287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:53:57.996287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:53:57.996287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:53:57.997232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:53:57.997232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:57.997232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:57.997232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:57.997232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:57.997232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:57.997232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:57.997232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:57.997232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:57.997232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:57.997232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:57.997232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:57.997232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:57.997232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:57.997232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:57.997232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:57.997232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:57.998147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:53:57.998147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:53:57.998147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:53:57.998147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:53:57.998147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:53:57.998147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:53:57.998147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:57.998147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:57.998147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 321/654] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:57.998627 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60164b-9866-4687-93aa-0ff0ead15670 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:58.017716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63f95e-43b8-45a2-a174-64392f94e527 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:58.050270 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c177650-6c50-4420-b762-b97fc3359ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:58.051066 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c177650-6c50-4420-b762-b97fc3359ded None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:58.051346 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c177650-6c50-4420-b762-b97fc3359ded None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:58.051621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c177650-6c50-4420-b762-b97fc3359ded None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:58.058817 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c177650-6c50-4420-b762-b97fc3359ded None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:58.060272 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c177650-6c50-4420-b762-b97fc3359ded None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:58.075921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-2c177650-6c50-4420-b762-b97fc3359ded None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 25 15:53:58.075921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:58.075921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:58.075921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:58.075921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:58.075921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:58.075921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:58.075921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:58.075921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:58.075921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:58.075921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:58.075921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:58.075921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:53:58.075921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:53:58.075921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:53:58.075921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:53:58.076909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:58.076909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:58.076909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:53:58.076909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:58.076909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:53:58.076909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:58.076909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:58.076909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:58.076909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:58.076909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:58.076909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:58.076909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:58.076909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:58.076909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:58.076909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:58.076909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:58.076909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:58.076909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:58.078158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:58.078158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:58.078158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:53:58.078158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:53:58.078158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:53:58.078158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:53:58.078158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:53:58.078158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:53:58.078158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 25 15:53:58.078158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:58.078158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 334/655] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:53:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:58.078839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b1fe3-3832-4f69-895a-f3a826ca2098 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:58.078839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b1fe3-3832-4f69-895a-f3a826ca2098 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:58.078839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b1fe3-3832-4f69-895a-f3a826ca2098 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:58.080228 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 335/656] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:53:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:53:58.082007 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b1fe3-3832-4f69-895a-f3a826ca2098 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:58.099859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:58.104719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e60164b-9866-4687-93aa-0ff0ead15670 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:58.140858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764a1fb-4065-467d-8e45-73c47d3b0637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:58.141972 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764a1fb-4065-467d-8e45-73c47d3b0637 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:58.142252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764a1fb-4065-467d-8e45-73c47d3b0637 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:58.142525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764a1fb-4065-467d-8e45-73c47d3b0637 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:58.152511 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764a1fb-4065-467d-8e45-73c47d3b0637 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:58.153996 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764a1fb-4065-467d-8e45-73c47d3b0637 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:58.171895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-d764a1fb-4065-467d-8e45-73c47d3b0637 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:58.171895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:58.171895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:58.171895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:58.171895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:58.171895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:58.171895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:58.171895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:58.171895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:58.171895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:58.171895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:58.171895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:58.171895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:58.171895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:58.171895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:58.172833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:58.172833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:58.172833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:58.172833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:58.172833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:58.172833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:58.172833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:58.172833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:58.172833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:58.172833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:58.172833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:58.172833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:58.172833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:58.172833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:58.172833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:58.172833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:58.172833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:58.174144 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:58.174144 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:58.174144 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:58.174144 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:58.174144 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 322/657] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:58.174698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764a1fb-4065-467d-8e45-73c47d3b0637 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:58.187896 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b1fe3-3832-4f69-895a-f3a826ca2098 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:58.201170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:58.222814 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd858838-9d73-401b-8dca-aebc3f418b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:58.223393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd858838-9d73-401b-8dca-aebc3f418b25 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:58.223674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd858838-9d73-401b-8dca-aebc3f418b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:58.223942 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd858838-9d73-401b-8dca-aebc3f418b25 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:58.224776 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd858838-9d73-401b-8dca-aebc3f418b25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:58.225080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd858838-9d73-401b-8dca-aebc3f418b25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:58.226512 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd858838-9d73-401b-8dca-aebc3f418b25 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:58.238175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 336/658] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:53:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:58.238919 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd858838-9d73-401b-8dca-aebc3f418b25 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:58.279274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764a1fb-4065-467d-8e45-73c47d3b0637 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:58.311943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48b7d7-529d-4adb-80e3-66cf8c71a6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:58.312630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48b7d7-529d-4adb-80e3-66cf8c71a6fe None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:58.312905 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48b7d7-529d-4adb-80e3-66cf8c71a6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:58.313170 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48b7d7-529d-4adb-80e3-66cf8c71a6fe None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:58.313957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48b7d7-529d-4adb-80e3-66cf8c71a6fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:58.314253 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48b7d7-529d-4adb-80e3-66cf8c71a6fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:58.315686 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48b7d7-529d-4adb-80e3-66cf8c71a6fe None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:58.327630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 323/659] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:53:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:58.328807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9956f-490c-4dd7-8aec-60c1ae96d9a2 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:58.329087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9956f-490c-4dd7-8aec-60c1ae96d9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:58.329360 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9956f-490c-4dd7-8aec-60c1ae96d9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:58.340314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd858838-9d73-401b-8dca-aebc3f418b25 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:58.345268 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-1cb9956f-490c-4dd7-8aec-60c1ae96d9a2 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:53:58.346434 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1cb9956f-490c-4dd7-8aec-60c1ae96d9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:53:58.349459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:58.373974 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f08d8-4505-4398-9b18-c96ddc1f41cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:58.374864 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f08d8-4505-4398-9b18-c96ddc1f41cd None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:58.375140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f08d8-4505-4398-9b18-c96ddc1f41cd None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:58.375411 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873f08d8-4505-4398-9b18-c96ddc1f41cd None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:58.376251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873f08d8-4505-4398-9b18-c96ddc1f41cd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:58.376594 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873f08d8-4505-4398-9b18-c96ddc1f41cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:58.378032 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873f08d8-4505-4398-9b18-c96ddc1f41cd None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:58.395038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 337/660] 10.222.0.13 () {56 vars in 1137 bytes} [Sat Jun 25 15:53:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:58.395827 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f08d8-4505-4398-9b18-c96ddc1f41cd None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:58.412847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 324/661] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:53:58.413551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9956f-490c-4dd7-8aec-60c1ae96d9a2 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:58.426906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:58.494311 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873f08d8-4505-4398-9b18-c96ddc1f41cd None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:58.511917 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9956f-490c-4dd7-8aec-60c1ae96d9a2 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:58.528849 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c26ac4-4afc-411f-9d41-1e4ac32fbbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:58.529513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c26ac4-4afc-411f-9d41-1e4ac32fbbe9 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:58.529793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c26ac4-4afc-411f-9d41-1e4ac32fbbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:58.530067 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c26ac4-4afc-411f-9d41-1e4ac32fbbe9 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:58.530781 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c26ac4-4afc-411f-9d41-1e4ac32fbbe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:58.531084 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c26ac4-4afc-411f-9d41-1e4ac32fbbe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:58.532464 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c26ac4-4afc-411f-9d41-1e4ac32fbbe9 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:58.543625 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 338/662] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:58.544403 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c26ac4-4afc-411f-9d41-1e4ac32fbbe9 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:58.544631 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968566c-9a01-475e-a6b0-cf417d92b8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:58.545348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968566c-9a01-475e-a6b0-cf417d92b8df None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:58.545627 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968566c-9a01-475e-a6b0-cf417d92b8df None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:58.545897 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968566c-9a01-475e-a6b0-cf417d92b8df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:58.546725 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968566c-9a01-475e-a6b0-cf417d92b8df None admin] RBAC: Adding query filter params (role.id=7f1996bcf7e54638b4f53b3d8ba83495, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=3622c5ccb7534e7c8e8342b4f6643bc3) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:58.547024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968566c-9a01-475e-a6b0-cf417d92b8df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:58.548611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968566c-9a01-475e-a6b0-cf417d92b8df None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:58.568454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 325/663] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:53:58 2022] GET /identity/v3/role_assignments?user.id=3622c5ccb7534e7c8e8342b4f6643bc3&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=7f1996bcf7e54638b4f53b3d8ba83495 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:58.568454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968566c-9a01-475e-a6b0-cf417d92b8df None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:58.568454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:58.642940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:58.657456 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c26ac4-4afc-411f-9d41-1e4ac32fbbe9 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:58.674185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968566c-9a01-475e-a6b0-cf417d92b8df None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:58.695154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cda3b-84e2-427d-a520-98209cdde547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:58.695883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cda3b-84e2-427d-a520-98209cdde547 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:58.696191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cda3b-84e2-427d-a520-98209cdde547 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:58.696688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cda3b-84e2-427d-a520-98209cdde547 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:58.697663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586cda3b-84e2-427d-a520-98209cdde547 None admin] RBAC: Adding query filter params (role.id=7f1996bcf7e54638b4f53b3d8ba83495, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=5cba086728824ec68fb0ee90fb3f99dd) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:58.697980 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586cda3b-84e2-427d-a520-98209cdde547 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:58.699440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586cda3b-84e2-427d-a520-98209cdde547 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:58.705790 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 339/664] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:53:58 2022] GET /identity/v3/role_assignments?user.id=5cba086728824ec68fb0ee90fb3f99dd&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=7f1996bcf7e54638b4f53b3d8ba83495 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:58.706593 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cda3b-84e2-427d-a520-98209cdde547 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:58.710658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f583dd-0b71-4a87-9454-9b4022cade3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:58.711305 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f583dd-0b71-4a87-9454-9b4022cade3c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:58.711641 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f583dd-0b71-4a87-9454-9b4022cade3c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:58.711916 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f583dd-0b71-4a87-9454-9b4022cade3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:58.801699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f583dd-0b71-4a87-9454-9b4022cade3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:58.803081 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f583dd-0b71-4a87-9454-9b4022cade3c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:58.806148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:58.851279 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cda3b-84e2-427d-a520-98209cdde547 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:58.897018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d92cd1-e57f-4fe8-ae4e-c03b9623c844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:58.897820 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d92cd1-e57f-4fe8-ae4e-c03b9623c844 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:58.898107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d92cd1-e57f-4fe8-ae4e-c03b9623c844 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:58.898383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d92cd1-e57f-4fe8-ae4e-c03b9623c844 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:58.910928 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d92cd1-e57f-4fe8-ae4e-c03b9623c844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:53:58.919587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-a2d92cd1-e57f-4fe8-ae4e-c03b9623c844 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:58.919587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:58.919587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:58.919587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:58.919587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:58.919587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:58.919587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:58.919587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:58.919587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:58.919587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:58.919587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:58.919587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:58.919587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:53:58.919587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:53:58.919587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:53:58.919587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:53:58.920567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:53:58.920567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:53:58.920567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:53:58.920567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:53:58.920567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:58.920567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:58.920567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:58.920567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:58.920567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:58.920567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:58.920567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:58.920567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:58.920567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:58.920567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:58.920567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:58.920567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:58.921404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:58.921404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:58.921404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:58.921404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:58.921404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:53:58.921404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:53:58.921404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:53:58.921404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:53:58.921404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:53:58.921404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:53:58.921404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:53:58.921404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:58.924567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 340/665] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:53:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:58.926218 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d92cd1-e57f-4fe8-ae4e-c03b9623c844 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:58.934899 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 326/666] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:58.935870 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f583dd-0b71-4a87-9454-9b4022cade3c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:58.947383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:59.047984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d92cd1-e57f-4fe8-ae4e-c03b9623c844 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:59.068638 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f583dd-0b71-4a87-9454-9b4022cade3c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:59.071953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:59.090927 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e57d84-18db-4c7e-b234-93895a0b0307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:59.092072 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e57d84-18db-4c7e-b234-93895a0b0307 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:59.092579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e57d84-18db-4c7e-b234-93895a0b0307 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:59.093063 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e57d84-18db-4c7e-b234-93895a0b0307 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:59.101650 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e57d84-18db-4c7e-b234-93895a0b0307 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:59.103895 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e57d84-18db-4c7e-b234-93895a0b0307 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:59.108954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d7c2d-3978-4423-9252-1618b4b776de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:59.109628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d7c2d-3978-4423-9252-1618b4b776de None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:59.109906 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d7c2d-3978-4423-9252-1618b4b776de None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:59.110176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d7c2d-3978-4423-9252-1618b4b776de None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:59.110918 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d7c2d-3978-4423-9252-1618b4b776de None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:59.111217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d7c2d-3978-4423-9252-1618b4b776de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:59.112908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d7c2d-3978-4423-9252-1618b4b776de None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:59.121225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-16e57d84-18db-4c7e-b234-93895a0b0307 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 25 15:53:59.121225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:59.121225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:59.121225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:59.121225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:59.121225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:59.121225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:59.121225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:59.121225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:59.121225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:59.121225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:59.121225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:59.121225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:53:59.121225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:53:59.121225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:53:59.121225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:53:59.122644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:59.122644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:59.122644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:53:59.122644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:59.122644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:53:59.122644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:59.122644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:59.122644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:59.122644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:59.122644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:59.122644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:59.122644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:59.122644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:59.122644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:59.122644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:59.122644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:59.122644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:59.122644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:59.123582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:59.123582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:59.123582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:53:59.123582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:53:59.123582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:53:59.123582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:53:59.123582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:53:59.123582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:53:59.123582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 25 15:53:59.123582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:59.124134 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 327/667] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:53:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:59.124134 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d7c2d-3978-4423-9252-1618b4b776de None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:59.126388 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 341/668] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:53:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:59.127655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e57d84-18db-4c7e-b234-93895a0b0307 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:59.211916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:59.239228 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e57d84-18db-4c7e-b234-93895a0b0307 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:59.259974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d7c2d-3978-4423-9252-1618b4b776de None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:59.275907 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d659ef-efad-4e1e-b456-1fb7c0c760c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:59.277256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d659ef-efad-4e1e-b456-1fb7c0c760c2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:59.277793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d659ef-efad-4e1e-b456-1fb7c0c760c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:59.278303 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d659ef-efad-4e1e-b456-1fb7c0c760c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:59.290163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d659ef-efad-4e1e-b456-1fb7c0c760c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:59.291982 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d659ef-efad-4e1e-b456-1fb7c0c760c2 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:59.302680 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77345a7-b56b-4e26-b536-5b947ca55858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:59.303485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77345a7-b56b-4e26-b536-5b947ca55858 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:59.303762 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77345a7-b56b-4e26-b536-5b947ca55858 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:59.304029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77345a7-b56b-4e26-b536-5b947ca55858 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:59.311621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-d4d659ef-efad-4e1e-b456-1fb7c0c760c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:59.311621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:59.311621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:59.311621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:59.311621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:59.311621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:59.311621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:59.311621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:59.311621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:59.311621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:59.311621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:59.311621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:59.311621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:59.311621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:59.311621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:59.312515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:59.312515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:59.312515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:59.312515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:59.312515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:59.312515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:59.312515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:59.312515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:59.312515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:59.312515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:59.312515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:59.312515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:59.312515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:59.312515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:59.312515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:59.312515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:59.312515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:59.313437 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:59.313437 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:59.313437 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:59.313437 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:59.315367 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77345a7-b56b-4e26-b536-5b947ca55858 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:59.315892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 342/669] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:59.316800 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77345a7-b56b-4e26-b536-5b947ca55858 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:59.317237 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d659ef-efad-4e1e-b456-1fb7c0c760c2 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:59.339661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-c77345a7-b56b-4e26-b536-5b947ca55858 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:59.339661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:59.339661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:59.339661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:59.339661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:59.339661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:59.339661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:59.339661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:59.339661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:59.339661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:59.339661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:59.339661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:59.339661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:53:59.339661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:53:59.339661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:53:59.340630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:53:59.340630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:59.340630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:59.340630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:59.340630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:59.340630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:59.340630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:59.340630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:59.340630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:59.340630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:59.340630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:59.340630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:59.340630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:59.340630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:59.340630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:59.340630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:59.340630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:59.342106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:53:59.342106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:53:59.342106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:53:59.342106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:53:59.342106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 328/670] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:59.342106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77345a7-b56b-4e26-b536-5b947ca55858 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:59.369087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:59.429667 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d659ef-efad-4e1e-b456-1fb7c0c760c2 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:59.456943 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:59.466539 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d43187-f861-49c3-b402-2922469f7b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:59.467584 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d43187-f861-49c3-b402-2922469f7b48 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:59.468105 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d43187-f861-49c3-b402-2922469f7b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:59.468607 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d43187-f861-49c3-b402-2922469f7b48 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:59.469703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d43187-f861-49c3-b402-2922469f7b48 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:59.470202 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d43187-f861-49c3-b402-2922469f7b48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:59.471830 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d43187-f861-49c3-b402-2922469f7b48 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:59.476945 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77345a7-b56b-4e26-b536-5b947ca55858 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:59.490701 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 343/671] 10.222.0.13 () {56 vars in 1139 bytes} [Sat Jun 25 15:53:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:59.491907 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d43187-f861-49c3-b402-2922469f7b48 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:59.518468 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683103c-bf16-44e5-8452-5babb9d0c309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:59.519140 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683103c-bf16-44e5-8452-5babb9d0c309 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:59.519469 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683103c-bf16-44e5-8452-5babb9d0c309 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:59.519688 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683103c-bf16-44e5-8452-5babb9d0c309 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:59.520434 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683103c-bf16-44e5-8452-5babb9d0c309 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:59.520753 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683103c-bf16-44e5-8452-5babb9d0c309 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:59.522230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6683103c-bf16-44e5-8452-5babb9d0c309 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:59.535544 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 329/672] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:59.536515 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683103c-bf16-44e5-8452-5babb9d0c309 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:59.597127 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d43187-f861-49c3-b402-2922469f7b48 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:59.621282 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:59.631865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c1f5a-c1f1-4692-9104-2c2c539dea8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:59.632687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c1f5a-c1f1-4692-9104-2c2c539dea8d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:59.633067 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c1f5a-c1f1-4692-9104-2c2c539dea8d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:59.633539 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c1f5a-c1f1-4692-9104-2c2c539dea8d None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:59.634871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c1f5a-c1f1-4692-9104-2c2c539dea8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:53:59.635535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c1f5a-c1f1-4692-9104-2c2c539dea8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:59.637084 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c1f5a-c1f1-4692-9104-2c2c539dea8d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:59.647442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6683103c-bf16-44e5-8452-5babb9d0c309 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:59.649665 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 344/673] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:53:59.650871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c1f5a-c1f1-4692-9104-2c2c539dea8d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:59.685643 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc13054-3be7-423b-a770-5fed7b508cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:59.686666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc13054-3be7-423b-a770-5fed7b508cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:59.687156 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc13054-3be7-423b-a770-5fed7b508cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:59.687632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc13054-3be7-423b-a770-5fed7b508cb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:59.697480 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:59.753405 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c1f5a-c1f1-4692-9104-2c2c539dea8d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:59.762958 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc13054-3be7-423b-a770-5fed7b508cb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:59.764835 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc13054-3be7-423b-a770-5fed7b508cb7 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:59.787863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72cae3b-a2d8-4b52-9176-ac260c72c955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:59.788457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cae3b-a2d8-4b52-9176-ac260c72c955 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:59.788742 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cae3b-a2d8-4b52-9176-ac260c72c955 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:59.789016 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72cae3b-a2d8-4b52-9176-ac260c72c955 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:59.796925 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72cae3b-a2d8-4b52-9176-ac260c72c955 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:59.798474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72cae3b-a2d8-4b52-9176-ac260c72c955 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:59.813169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-e72cae3b-a2d8-4b52-9176-ac260c72c955 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 25 15:53:59.813169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:59.813169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:59.813169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:59.813169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:59.813169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:59.813169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:59.813169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:59.813169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:59.813169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:59.813169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:59.813169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:59.813169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:53:59.813169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:53:59.813169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:53:59.813169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:53:59.814205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:59.814205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:59.814205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:53:59.814205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:59.814205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:53:59.814205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:53:59.814205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:59.814205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:59.814205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:59.814205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:59.814205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:59.814205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:59.814205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:59.814205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:59.814205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:59.814205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:59.814205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:59.814205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:59.815168 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:59.815168 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:59.815168 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:53:59.815168 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:53:59.815168 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:53:59.815168 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:53:59.815168 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:53:59.815168 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:53:59.815168 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 25 15:53:59.815168 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:59.815168 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 345/674] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:53:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:53:59.815774 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72cae3b-a2d8-4b52-9176-ac260c72c955 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:59.862840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:59.887900 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 330/675] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:53:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:53:59.889191 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc13054-3be7-423b-a770-5fed7b508cb7 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:53:59.932753 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72cae3b-a2d8-4b52-9176-ac260c72c955 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:53:59.934976 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:53:59.969900 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ad2ab-868e-4680-a503-73046f6abeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:53:59.970719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ad2ab-868e-4680-a503-73046f6abeed None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:53:59.970996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ad2ab-868e-4680-a503-73046f6abeed None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:53:59.971279 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ad2ab-868e-4680-a503-73046f6abeed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:53:59.978998 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ad2ab-868e-4680-a503-73046f6abeed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:53:59.980512 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ad2ab-868e-4680-a503-73046f6abeed None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:53:59.994746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-920ad2ab-868e-4680-a503-73046f6abeed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:59.994746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:53:59.994746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:53:59.994746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:53:59.994746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:53:59.994746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:53:59.994746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:53:59.994746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:53:59.994746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:53:59.994746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:53:59.994746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:53:59.994746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:53:59.994746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:53:59.994746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:53:59.994746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:53:59.995672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:53:59.995672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:53:59.995672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:53:59.995672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:53:59.995672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:53:59.995672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:53:59.995672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:53:59.995672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:53:59.995672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:53:59.995672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:53:59.995672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:53:59.995672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:53:59.995672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:53:59.995672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:53:59.995672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:53:59.995672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:53:59.995672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:53:59.996578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:53:59.996578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:53:59.996578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:53:59.996578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:53:59.996578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:53:59.996578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:53:59.996578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:53:59.996578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:53:59.999215 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc13054-3be7-423b-a770-5fed7b508cb7 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:00.000845 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 346/676] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:53:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:00.001692 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ad2ab-868e-4680-a503-73046f6abeed None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:00.036389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1538a2f-f894-434b-9ede-9c42152f96e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:00.037441 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1538a2f-f894-434b-9ede-9c42152f96e5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:00.037992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1538a2f-f894-434b-9ede-9c42152f96e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:00.038461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1538a2f-f894-434b-9ede-9c42152f96e5 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:00.039659 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1538a2f-f894-434b-9ede-9c42152f96e5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:00.040163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1538a2f-f894-434b-9ede-9c42152f96e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:00.041829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1538a2f-f894-434b-9ede-9c42152f96e5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:00.064889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 331/677] 10.222.0.13 () {56 vars in 1135 bytes} [Sat Jun 25 15:53:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:00.066007 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1538a2f-f894-434b-9ede-9c42152f96e5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:00.101518 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:00.105175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ad2ab-868e-4680-a503-73046f6abeed None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:00.139563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4272c-052f-4e7f-9a53-a33c56852cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:00.140926 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4272c-052f-4e7f-9a53-a33c56852cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:00.140926 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4272c-052f-4e7f-9a53-a33c56852cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:00.140926 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4272c-052f-4e7f-9a53-a33c56852cf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:00.151715 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4272c-052f-4e7f-9a53-a33c56852cf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:00.153173 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4272c-052f-4e7f-9a53-a33c56852cf1 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:00.170651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-41c4272c-052f-4e7f-9a53-a33c56852cf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:00.170651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:00.170651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:00.170651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:00.170651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:00.170651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:00.170651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:00.170651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:00.170651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:00.170651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:00.170651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:00.170651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:00.170651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:00.170651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:00.170651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:00.171583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:00.171583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:00.171583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:00.171583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:00.171583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:00.171583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:00.171583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:00.171583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:00.171583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:00.171583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:00.171583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:00.171583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:00.171583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:00.171583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:00.171583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:00.171583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:00.171583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:00.172491 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:00.172491 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:00.172491 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:00.172491 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:00.172491 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 347/678] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:00.173228 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4272c-052f-4e7f-9a53-a33c56852cf1 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:00.175286 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1538a2f-f894-434b-9ede-9c42152f96e5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:00.191748 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:00.211370 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd5607-3ce8-4536-a89a-5fc234ad567c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:00.212489 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd5607-3ce8-4536-a89a-5fc234ad567c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:00.212823 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd5607-3ce8-4536-a89a-5fc234ad567c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:00.213193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd5607-3ce8-4536-a89a-5fc234ad567c None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:00.214086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd5607-3ce8-4536-a89a-5fc234ad567c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:00.214493 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd5607-3ce8-4536-a89a-5fc234ad567c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:00.216018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd5607-3ce8-4536-a89a-5fc234ad567c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:00.232990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 332/679] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:54:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:00.232990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd5607-3ce8-4536-a89a-5fc234ad567c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:00.298112 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4272c-052f-4e7f-9a53-a33c56852cf1 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:00.334547 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e7205-756f-4e5e-812c-a684314ef37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:00.335607 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e7205-756f-4e5e-812c-a684314ef37c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:00.336096 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e7205-756f-4e5e-812c-a684314ef37c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:00.336820 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e7205-756f-4e5e-812c-a684314ef37c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:00.353509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:00.361101 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd5607-3ce8-4536-a89a-5fc234ad567c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:00.398595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe769c2f-1a92-473b-a348-463d055c8fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:00.399316 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe769c2f-1a92-473b-a348-463d055c8fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:00.399563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe769c2f-1a92-473b-a348-463d055c8fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:00.399848 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe769c2f-1a92-473b-a348-463d055c8fc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:00.400599 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe769c2f-1a92-473b-a348-463d055c8fc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:00.400903 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe769c2f-1a92-473b-a348-463d055c8fc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:00.402611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe769c2f-1a92-473b-a348-463d055c8fc9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:00.409507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e7205-756f-4e5e-812c-a684314ef37c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:00.411244 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e7205-756f-4e5e-812c-a684314ef37c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:00.416096 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 333/680] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:00.417059 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe769c2f-1a92-473b-a348-463d055c8fc9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:00.430551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:00.519759 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 348/681] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:54:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:54:00.521013 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e7205-756f-4e5e-812c-a684314ef37c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:00.526750 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe769c2f-1a92-473b-a348-463d055c8fc9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:00.563892 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a32a83-f229-4e0b-9882-6f04456af829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:00.564874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a32a83-f229-4e0b-9882-6f04456af829 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:00.565005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a32a83-f229-4e0b-9882-6f04456af829 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:00.565236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a32a83-f229-4e0b-9882-6f04456af829 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:00.566075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a32a83-f229-4e0b-9882-6f04456af829 None admin] RBAC: Adding query filter params (role.id=7f1996bcf7e54638b4f53b3d8ba83495, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=b00152c72a1f4755b09c1dbd1d874b11) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:00.566381 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a32a83-f229-4e0b-9882-6f04456af829 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:00.567972 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a32a83-f229-4e0b-9882-6f04456af829 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:00.580454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 334/682] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:54:00 2022] GET /identity/v3/role_assignments?user.id=b00152c72a1f4755b09c1dbd1d874b11&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=7f1996bcf7e54638b4f53b3d8ba83495 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:00.580454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a32a83-f229-4e0b-9882-6f04456af829 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:00.609069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:00.702296 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a32a83-f229-4e0b-9882-6f04456af829 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:00.724149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e7205-756f-4e5e-812c-a684314ef37c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:00.748213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:00.750558 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf87a2-f757-4d47-9636-b42b6269893c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:00.751359 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf87a2-f757-4d47-9636-b42b6269893c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:00.751641 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf87a2-f757-4d47-9636-b42b6269893c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:00.751914 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf87a2-f757-4d47-9636-b42b6269893c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:00.765801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf87a2-f757-4d47-9636-b42b6269893c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:00.767607 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf87a2-f757-4d47-9636-b42b6269893c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:00.776286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b398f-6f1a-4f3b-8f86-d7b683de984c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:00.777253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b398f-6f1a-4f3b-8f86-d7b683de984c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:00.777922 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b398f-6f1a-4f3b-8f86-d7b683de984c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:00.778207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b398f-6f1a-4f3b-8f86-d7b683de984c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:00.785671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b398f-6f1a-4f3b-8f86-d7b683de984c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:00.787042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-f4bf87a2-f757-4d47-9636-b42b6269893c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:00.787042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:00.787042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:00.787042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:00.787042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:00.787042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:00.787042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:00.787042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:00.787042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:00.787042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:00.787042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:00.787042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:00.787042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:00.787042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:00.787042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:00.787974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:00.787974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:00.787974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:00.787974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:00.787974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:00.787974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:00.787974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:00.787974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:00.787974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:00.787974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:00.787974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:00.787974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:00.787974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:00.787974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:00.787974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:00.787974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:00.787974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:00.788890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:00.788890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:00.788890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:00.788890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:00.789126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b398f-6f1a-4f3b-8f86-d7b683de984c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:00.789198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 335/683] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:00.789923 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139abe0b-08bb-41c8-861c-484e4af72f40 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:00.790201 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139abe0b-08bb-41c8-861c-484e4af72f40 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:00.790472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139abe0b-08bb-41c8-861c-484e4af72f40 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:00.795847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 336/684] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:00.796575 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139abe0b-08bb-41c8-861c-484e4af72f40 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:00.804648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-b51b398f-6f1a-4f3b-8f86-d7b683de984c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:00.804648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:00.804648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:00.804648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:00.804648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:00.804648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:00.804648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:00.804648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:00.804648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:00.804648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:00.804648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:00.804648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:00.804648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:54:00.804648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:54:00.804648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:54:00.805616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:54:00.805616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:00.805616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:00.805616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:00.805616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:00.805616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:00.805616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:00.805616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:00.805616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:00.805616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:00.805616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:00.805616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:00.805616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:00.805616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:00.805616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:00.805616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:00.805616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:00.806504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:54:00.806504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:54:00.806504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:54:00.806504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:54:00.806504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:54:00.806504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:54:00.806504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:00.806504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:00.809116 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 349/685] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:00.810675 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605c56a-4214-4e7d-8ed1-d1f0fd26fdc4 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:00.810968 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605c56a-4214-4e7d-8ed1-d1f0fd26fdc4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:00.811246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605c56a-4214-4e7d-8ed1-d1f0fd26fdc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:00.829217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-0605c56a-4214-4e7d-8ed1-d1f0fd26fdc4 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:00.830414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0605c56a-4214-4e7d-8ed1-d1f0fd26fdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:00.891682 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:00.916497 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 350/686] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:54:00.917893 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605c56a-4214-4e7d-8ed1-d1f0fd26fdc4 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:00.944632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139abe0b-08bb-41c8-861c-484e4af72f40 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:00.981325 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3e7ee-842b-4a57-a951-e345a96befaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:00.981913 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3e7ee-842b-4a57-a951-e345a96befaf None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:00.982193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3e7ee-842b-4a57-a951-e345a96befaf None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:00.982473 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3e7ee-842b-4a57-a951-e345a96befaf None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:00.983202 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3e7ee-842b-4a57-a951-e345a96befaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:00.983587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3e7ee-842b-4a57-a951-e345a96befaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:00.984999 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3e7ee-842b-4a57-a951-e345a96befaf None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:01.000540 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 337/687] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:01.000540 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3e7ee-842b-4a57-a951-e345a96befaf None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:01.037398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605c56a-4214-4e7d-8ed1-d1f0fd26fdc4 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:01.045103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:01.074552 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c629c2-d412-4def-ac00-22111acb559b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:01.075224 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c629c2-d412-4def-ac00-22111acb559b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:01.075507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c629c2-d412-4def-ac00-22111acb559b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:01.075775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c629c2-d412-4def-ac00-22111acb559b None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:01.076597 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c629c2-d412-4def-ac00-22111acb559b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:01.076904 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c629c2-d412-4def-ac00-22111acb559b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:01.078329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c629c2-d412-4def-ac00-22111acb559b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:01.089979 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 351/688] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:54:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:01.091242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c01077-d954-479c-8433-d007f11a8de0 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:01.091521 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c01077-d954-479c-8433-d007f11a8de0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:01.091792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c01077-d954-479c-8433-d007f11a8de0 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:01.093193 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 352/689] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:01.093724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c01077-d954-479c-8433-d007f11a8de0 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:01.116885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3e7ee-842b-4a57-a951-e345a96befaf None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:01.158122 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:01.165818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28382a73-af67-493d-bad5-f6691a69eb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:01.167100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28382a73-af67-493d-bad5-f6691a69eb68 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:01.167568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28382a73-af67-493d-bad5-f6691a69eb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:01.168026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28382a73-af67-493d-bad5-f6691a69eb68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:01.181315 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28382a73-af67-493d-bad5-f6691a69eb68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:54:01.191817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-28382a73-af67-493d-bad5-f6691a69eb68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:54:01.191817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:01.191817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:01.191817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:01.191817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:01.191817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:01.191817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:01.191817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:01.191817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:01.191817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:01.191817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:01.191817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:01.191817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:54:01.191817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:54:01.191817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:54:01.191817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:54:01.192774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:54:01.192774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:54:01.192774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:54:01.192774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:54:01.192774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:01.192774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:01.192774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:01.192774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:01.192774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:01.192774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:01.192774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:01.192774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:01.192774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:01.192774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:01.192774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:01.192774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:01.193628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:01.193628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:01.193628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:01.193628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:01.193628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:54:01.193628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:54:01.193628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:54:01.193628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:54:01.193628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:54:01.193628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:54:01.193628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:54:01.193628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:01.194274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 338/690] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:54:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:01.195018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28382a73-af67-493d-bad5-f6691a69eb68 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:01.216637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c01077-d954-479c-8433-d007f11a8de0 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:01.251702 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19733ce3-fd2d-44ba-a4d2-73bf063d629b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:01.252442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19733ce3-fd2d-44ba-a4d2-73bf063d629b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:01.252724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19733ce3-fd2d-44ba-a4d2-73bf063d629b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:01.252990 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19733ce3-fd2d-44ba-a4d2-73bf063d629b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:01.260394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19733ce3-fd2d-44ba-a4d2-73bf063d629b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:01.261916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19733ce3-fd2d-44ba-a4d2-73bf063d629b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:01.276557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-19733ce3-fd2d-44ba-a4d2-73bf063d629b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:01.276557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:01.276557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:01.276557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:01.276557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:01.276557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:01.276557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:01.276557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:01.276557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:01.276557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:01.276557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:01.276557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:01.276557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:54:01.276557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:54:01.276557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:54:01.277671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:54:01.277671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:01.277671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:01.277671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:01.277671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:01.277671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:01.277671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:01.277671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:01.277671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:01.277671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:01.277671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:01.277671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:01.277671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:01.277671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:01.277671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:01.277671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:01.277671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:01.278591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:54:01.278591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:54:01.278591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:54:01.278591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:54:01.278591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:54:01.278591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:54:01.278591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:01.278591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:01.278591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 353/691] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:01.279338 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7b5bc-33b5-4916-bcaa-656e05866af8 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:01.279624 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7b5bc-33b5-4916-bcaa-656e05866af8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:01.279901 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7b5bc-33b5-4916-bcaa-656e05866af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:01.297238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a8d7b5bc-33b5-4916-bcaa-656e05866af8 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:01.298406 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a8d7b5bc-33b5-4916-bcaa-656e05866af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:01.306915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28382a73-af67-493d-bad5-f6691a69eb68 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:01.318800 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:01.344329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c81a8-eca8-4b6b-9230-81c567c0e132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:01.345011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c81a8-eca8-4b6b-9230-81c567c0e132 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:01.345289 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c81a8-eca8-4b6b-9230-81c567c0e132 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:01.345558 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c81a8-eca8-4b6b-9230-81c567c0e132 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:01.346384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c81a8-eca8-4b6b-9230-81c567c0e132 None admin] RBAC: Adding query filter params (role.id=7f1996bcf7e54638b4f53b3d8ba83495, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=aaffb9f0bf714822829455c0487376ca) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:01.346692 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c81a8-eca8-4b6b-9230-81c567c0e132 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:01.348126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c81a8-eca8-4b6b-9230-81c567c0e132 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:01.364508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 339/692] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:54:01 2022] GET /identity/v3/role_assignments?user.id=aaffb9f0bf714822829455c0487376ca&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=7f1996bcf7e54638b4f53b3d8ba83495 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:01.364508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c81a8-eca8-4b6b-9230-81c567c0e132 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:01.388457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 354/693] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:54:01.389597 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7b5bc-33b5-4916-bcaa-656e05866af8 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:01.457754 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:01.514106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c81a8-eca8-4b6b-9230-81c567c0e132 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:01.547410 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7b5bc-33b5-4916-bcaa-656e05866af8 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:01.561303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9ab23-2cc9-4e34-88f1-af98f78c9274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:01.561955 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9ab23-2cc9-4e34-88f1-af98f78c9274 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:01.562249 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9ab23-2cc9-4e34-88f1-af98f78c9274 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:01.562522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9ab23-2cc9-4e34-88f1-af98f78c9274 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:01.563277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9ab23-2cc9-4e34-88f1-af98f78c9274 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:01.563578 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9ab23-2cc9-4e34-88f1-af98f78c9274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:01.565053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9ab23-2cc9-4e34-88f1-af98f78c9274 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:01.576192 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 340/694] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:54:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:01.577045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9ab23-2cc9-4e34-88f1-af98f78c9274 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:01.592291 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e089d0c-ae10-4947-bca9-632c3bf447d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:01.593167 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e089d0c-ae10-4947-bca9-632c3bf447d3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:01.593549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e089d0c-ae10-4947-bca9-632c3bf447d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:01.593932 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e089d0c-ae10-4947-bca9-632c3bf447d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:01.594833 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e089d0c-ae10-4947-bca9-632c3bf447d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:01.595246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e089d0c-ae10-4947-bca9-632c3bf447d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:01.597678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e089d0c-ae10-4947-bca9-632c3bf447d3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:01.610102 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 355/695] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:54:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:01.611755 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e089d0c-ae10-4947-bca9-632c3bf447d3 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:01.632000 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:01.723352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9ab23-2cc9-4e34-88f1-af98f78c9274 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:01.724641 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e089d0c-ae10-4947-bca9-632c3bf447d3 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:01.744272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:01.762148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5ebf0-10bd-4eb0-8dd2-747a72c28fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:01.763250 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5ebf0-10bd-4eb0-8dd2-747a72c28fbf None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:01.763671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5ebf0-10bd-4eb0-8dd2-747a72c28fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:01.764078 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5ebf0-10bd-4eb0-8dd2-747a72c28fbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:01.770999 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07613467-65cc-4379-9746-b66823c93f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:01.771917 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07613467-65cc-4379-9746-b66823c93f25 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:01.772303 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07613467-65cc-4379-9746-b66823c93f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:01.772709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07613467-65cc-4379-9746-b66823c93f25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:01.778539 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc5ebf0-10bd-4eb0-8dd2-747a72c28fbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:01.780213 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc5ebf0-10bd-4eb0-8dd2-747a72c28fbf None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:01.786792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07613467-65cc-4379-9746-b66823c93f25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:01.788799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07613467-65cc-4379-9746-b66823c93f25 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:01.798579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-adc5ebf0-10bd-4eb0-8dd2-747a72c28fbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:01.798579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:01.798579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:01.798579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:01.798579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:01.798579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:01.798579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:01.798579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:01.798579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:01.798579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:01.798579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:01.798579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:01.798579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:01.798579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:01.798579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:01.799666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:01.799666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:01.799666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:01.799666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:01.799666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:01.799666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:01.799666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:01.799666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:01.799666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:01.799666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:01.799666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:01.799666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:01.799666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:01.799666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:01.799666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:01.799666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:01.799666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:01.800786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:01.800786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:01.800786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:01.800786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:01.802683 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 341/696] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:01.803629 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5ebf0-10bd-4eb0-8dd2-747a72c28fbf None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:01.810967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-07613467-65cc-4379-9746-b66823c93f25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:01.810967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:01.810967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:01.810967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:01.810967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:01.810967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:01.810967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:01.810967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:01.810967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:01.810967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:01.810967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:01.810967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:01.810967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:01.810967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:01.810967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:01.812100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:01.812100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:01.812100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:01.812100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:01.812100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:01.812100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:01.812100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:01.812100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:01.812100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:01.812100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:01.812100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:01.812100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:01.812100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:01.812100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:01.812100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:01.812100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:01.812100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:01.813266 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:01.813266 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:01.813266 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:01.813266 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:01.814796 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 356/697] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:01.815666 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07613467-65cc-4379-9746-b66823c93f25 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:01.905669 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:01.913816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5ebf0-10bd-4eb0-8dd2-747a72c28fbf None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:01.949768 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b367c-9f70-455c-91fd-be5652746d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:01.950100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07613467-65cc-4379-9746-b66823c93f25 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:01.950851 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b367c-9f70-455c-91fd-be5652746d4a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:01.951273 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b367c-9f70-455c-91fd-be5652746d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:01.951710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b367c-9f70-455c-91fd-be5652746d4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:01.952845 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b367c-9f70-455c-91fd-be5652746d4a None admin] RBAC: Adding query filter params (role.id=7f1996bcf7e54638b4f53b3d8ba83495, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=19788c4239ca44f9a31a7d32086c8947) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:01.953301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b367c-9f70-455c-91fd-be5652746d4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:01.954857 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b367c-9f70-455c-91fd-be5652746d4a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:01.968890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 342/698] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:54:01 2022] GET /identity/v3/role_assignments?user.id=19788c4239ca44f9a31a7d32086c8947&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=7f1996bcf7e54638b4f53b3d8ba83495 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:01.970214 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b367c-9f70-455c-91fd-be5652746d4a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:02.003352 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf43c9a-d3cb-4aff-a1f0-f8169f11b2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:02.004254 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf43c9a-d3cb-4aff-a1f0-f8169f11b2fc None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:02.004677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf43c9a-d3cb-4aff-a1f0-f8169f11b2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:02.005051 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf43c9a-d3cb-4aff-a1f0-f8169f11b2fc None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:02.005911 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf43c9a-d3cb-4aff-a1f0-f8169f11b2fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:02.006332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf43c9a-d3cb-4aff-a1f0-f8169f11b2fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:02.007875 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf43c9a-d3cb-4aff-a1f0-f8169f11b2fc None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:02.038963 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 357/699] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:02.044713 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf43c9a-d3cb-4aff-a1f0-f8169f11b2fc None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:02.048827 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:02.174244 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf43c9a-d3cb-4aff-a1f0-f8169f11b2fc None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:02.191022 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b367c-9f70-455c-91fd-be5652746d4a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:02.205344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:02.219724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e7693-c825-4280-9109-fb7d2d820428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:02.220449 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e7693-c825-4280-9109-fb7d2d820428 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:02.220864 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e7693-c825-4280-9109-fb7d2d820428 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:02.221146 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e7693-c825-4280-9109-fb7d2d820428 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:02.222207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e7693-c825-4280-9109-fb7d2d820428 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:02.223906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e7693-c825-4280-9109-fb7d2d820428 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:02.234032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d068abe-cc34-4fb7-9cfa-cb48f10b1b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:02.235138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d068abe-cc34-4fb7-9cfa-cb48f10b1b8a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:02.235658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d068abe-cc34-4fb7-9cfa-cb48f10b1b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:02.236202 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d068abe-cc34-4fb7-9cfa-cb48f10b1b8a None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:02.237324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d068abe-cc34-4fb7-9cfa-cb48f10b1b8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:02.237928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d068abe-cc34-4fb7-9cfa-cb48f10b1b8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:02.239779 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d068abe-cc34-4fb7-9cfa-cb48f10b1b8a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:02.249241 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 358/700] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Jun 25 15:54:02 2022] POST /identity/v3/users => generated 254 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:54:02.258664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 343/701] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:02.264884 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e7693-c825-4280-9109-fb7d2d820428 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:02.384941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e7693-c825-4280-9109-fb7d2d820428 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:02.425909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2f506-4449-4d02-ad33-4cf39d0cf844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:02.426825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2f506-4449-4d02-ad33-4cf39d0cf844 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:02.427106 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2f506-4449-4d02-ad33-4cf39d0cf844 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:02.427376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2f506-4449-4d02-ad33-4cf39d0cf844 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:02.436831 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2f506-4449-4d02-ad33-4cf39d0cf844 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:02.438293 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2f506-4449-4d02-ad33-4cf39d0cf844 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:02.452677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-88f2f506-4449-4d02-ad33-4cf39d0cf844 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 25 15:54:02.452677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:02.452677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:02.452677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:02.452677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:02.452677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:02.452677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:02.452677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:02.452677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:02.452677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:02.452677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:02.452677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:02.452677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:54:02.452677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:54:02.452677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:54:02.452677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:54:02.453659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:02.453659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:02.453659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:54:02.453659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:02.453659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:54:02.453659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:02.453659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:02.453659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:02.453659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:02.453659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:02.453659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:02.453659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:02.453659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:02.453659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:02.453659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:02.453659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:02.453659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:02.453659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:02.454658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:02.454658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:02.454658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:54:02.454658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:54:02.454658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:54:02.454658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:54:02.454658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:54:02.454658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:54:02.454658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 25 15:54:02.454658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:02.454658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 359/702] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Jun 25 15:54:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:02.461682 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d068abe-cc34-4fb7-9cfa-cb48f10b1b8a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:02.558342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:02.566677 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d068abe-cc34-4fb7-9cfa-cb48f10b1b8a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:02.615538 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d2486-484c-4b3f-9971-942ad39ac772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:02.616552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d2486-484c-4b3f-9971-942ad39ac772 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:02.616986 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d2486-484c-4b3f-9971-942ad39ac772 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:02.617397 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d2486-484c-4b3f-9971-942ad39ac772 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:02.618825 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d2486-484c-4b3f-9971-942ad39ac772 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:02.619134 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d2486-484c-4b3f-9971-942ad39ac772 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:02.620681 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d2486-484c-4b3f-9971-942ad39ac772 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:02.647708 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 344/703] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Jun 25 15:54:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:02.653660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2f506-4449-4d02-ad33-4cf39d0cf844 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:02.709834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:02.780159 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2f506-4449-4d02-ad33-4cf39d0cf844 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:02.827007 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1127d-d9a8-49fa-a84d-ab39026387c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:02.827671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1127d-d9a8-49fa-a84d-ab39026387c4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:02.827956 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1127d-d9a8-49fa-a84d-ab39026387c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:02.828236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1127d-d9a8-49fa-a84d-ab39026387c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:02.831615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09945b-6210-4185-9979-258a26900af2 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:02.831903 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09945b-6210-4185-9979-258a26900af2 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:02.832178 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa09945b-6210-4185-9979-258a26900af2 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:02.833858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 345/704] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:02.843789 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b80d3-3416-412b-a9b5-94479e4e0f62 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:02.844074 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b80d3-3416-412b-a9b5-94479e4e0f62 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:02.844352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b80d3-3416-412b-a9b5-94479e4e0f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:02.873878 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f00b80d3-3416-412b-a9b5-94479e4e0f62 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:02.875445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f00b80d3-3416-412b-a9b5-94479e4e0f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:02.923659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1127d-d9a8-49fa-a84d-ab39026387c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:02.925194 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1127d-d9a8-49fa-a84d-ab39026387c4 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:02.978722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:02.996975 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 346/705] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:54:03.023366 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b80d3-3416-412b-a9b5-94479e4e0f62 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:03.099343 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 360/706] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:54:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:54:03.110023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1127d-d9a8-49fa-a84d-ab39026387c4 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:03.122901 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:03.141142 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b80d3-3416-412b-a9b5-94479e4e0f62 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:03.180225 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba371f-1ae4-4dae-8d75-9975dc207f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:03.181763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba371f-1ae4-4dae-8d75-9975dc207f2f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:03.182056 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba371f-1ae4-4dae-8d75-9975dc207f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:03.182637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba371f-1ae4-4dae-8d75-9975dc207f2f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:03.183095 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba371f-1ae4-4dae-8d75-9975dc207f2f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:03.184549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba371f-1ae4-4dae-8d75-9975dc207f2f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:03.199557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-05ba371f-1ae4-4dae-8d75-9975dc207f2f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 25 15:54:03.199557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:03.199557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:03.199557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:03.199557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:03.199557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:03.199557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:03.199557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:03.199557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 25 15:54:03.199557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:03.199557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:03.199557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:03.199557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:03.199557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:03.199557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:03.199557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:03.200583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:03.200583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:03.200583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:03.200583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:03.200583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:03.200583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:03.200583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:03.200583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:03.200583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:03.200583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:03.200583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:03.200583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:03.200583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:03.200583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:03.200583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:03.200583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:03.201430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:03.201430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:03.201430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:03.201430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:03.201430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:03.201430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:03.201430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:03.201430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:03.201430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:03.201430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:03.201430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:03.201430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:03.201430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:03.201430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 25 15:54:03.201430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:03.201430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 347/707] 10.222.0.13 () {56 vars in 1129 bytes} [Sat Jun 25 15:54:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:03.206620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba371f-1ae4-4dae-8d75-9975dc207f2f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:03.320433 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba371f-1ae4-4dae-8d75-9975dc207f2f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:03.338445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1127d-d9a8-49fa-a84d-ab39026387c4 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:03.356157 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afcc8e-262d-4c7e-b9c5-e69abe58de4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:03.356948 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afcc8e-262d-4c7e-b9c5-e69abe58de4c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:03.357234 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afcc8e-262d-4c7e-b9c5-e69abe58de4c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:03.357512 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afcc8e-262d-4c7e-b9c5-e69abe58de4c None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:03.358235 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73afcc8e-262d-4c7e-b9c5-e69abe58de4c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:03.358551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73afcc8e-262d-4c7e-b9c5-e69abe58de4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:03.359941 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73afcc8e-262d-4c7e-b9c5-e69abe58de4c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:03.368549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 348/708] 10.222.0.13 () {56 vars in 1124 bytes} [Sat Jun 25 15:54:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:03.377620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afcc8e-262d-4c7e-b9c5-e69abe58de4c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:03.395610 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:03.397327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030fb6c-7844-46eb-847e-1a6dc3a1575b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:03.398391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030fb6c-7844-46eb-847e-1a6dc3a1575b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:03.398937 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030fb6c-7844-46eb-847e-1a6dc3a1575b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:03.399505 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030fb6c-7844-46eb-847e-1a6dc3a1575b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:03.420445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030fb6c-7844-46eb-847e-1a6dc3a1575b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:03.422382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030fb6c-7844-46eb-847e-1a6dc3a1575b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:03.427555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:03.457644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-e030fb6c-7844-46eb-847e-1a6dc3a1575b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:03.457644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:03.457644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:03.457644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:03.457644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:03.457644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:03.457644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:03.457644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:03.457644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:03.457644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:03.457644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:03.457644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:03.457644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:03.457644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:03.457644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:03.458561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:03.458561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:03.458561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:03.458561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:03.458561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:03.458561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:03.458561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:03.458561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:03.458561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:03.458561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:03.458561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:03.458561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:03.458561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:03.458561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:03.458561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:03.458561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:03.458561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:03.459449 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:03.459449 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:03.459449 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:03.459449 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:03.459449 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 361/709] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:03.464442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030fb6c-7844-46eb-847e-1a6dc3a1575b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:03.491884 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afcc8e-262d-4c7e-b9c5-e69abe58de4c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:03.561099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43139c-31a5-40ea-921e-fe8e37cecff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:03.561769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43139c-31a5-40ea-921e-fe8e37cecff3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:03.562062 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43139c-31a5-40ea-921e-fe8e37cecff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:03.562374 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43139c-31a5-40ea-921e-fe8e37cecff3 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:03.563087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43139c-31a5-40ea-921e-fe8e37cecff3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:03.563384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43139c-31a5-40ea-921e-fe8e37cecff3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:03.564796 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43139c-31a5-40ea-921e-fe8e37cecff3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:03.571272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 349/710] 10.222.0.13 () {56 vars in 1124 bytes} [Sat Jun 25 15:54:03 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:03.657682 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030fb6c-7844-46eb-847e-1a6dc3a1575b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:03.708072 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f3b89-e0fc-4bdf-a3c8-6860fa0f573b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:03.708728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f3b89-e0fc-4bdf-a3c8-6860fa0f573b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:03.709014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f3b89-e0fc-4bdf-a3c8-6860fa0f573b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:03.709295 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f3b89-e0fc-4bdf-a3c8-6860fa0f573b None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:03.710032 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f3b89-e0fc-4bdf-a3c8-6860fa0f573b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:03.710342 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f3b89-e0fc-4bdf-a3c8-6860fa0f573b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:03.711944 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f3b89-e0fc-4bdf-a3c8-6860fa0f573b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:03.728942 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 362/711] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:03.734169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43139c-31a5-40ea-921e-fe8e37cecff3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:03.812761 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646f6a1-387f-4823-a0f6-224ae08b2d3c None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:03.813109 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646f6a1-387f-4823-a0f6-224ae08b2d3c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:03.813325 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646f6a1-387f-4823-a0f6-224ae08b2d3c None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:03.814958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 363/712] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:03.821013 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516c869-c3ea-4337-9218-efef3ad0a7ba None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:03.821307 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516c869-c3ea-4337-9218-efef3ad0a7ba None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:03.821594 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516c869-c3ea-4337-9218-efef3ad0a7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:03.842029 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f516c869-c3ea-4337-9218-efef3ad0a7ba None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:03.843354 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f516c869-c3ea-4337-9218-efef3ad0a7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:03.869119 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:03.897901 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43139c-31a5-40ea-921e-fe8e37cecff3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:03.904816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:03.926506 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 364/713] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:54:03.936797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7729d8f8-e2df-44db-816a-fe21e480ad59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:03.937596 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729d8f8-e2df-44db-816a-fe21e480ad59 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:03.937872 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729d8f8-e2df-44db-816a-fe21e480ad59 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:03.938140 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7729d8f8-e2df-44db-816a-fe21e480ad59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:03.945033 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7729d8f8-e2df-44db-816a-fe21e480ad59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:03.946613 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7729d8f8-e2df-44db-816a-fe21e480ad59 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:03.950659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516c869-c3ea-4337-9218-efef3ad0a7ba None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:03.974304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-7729d8f8-e2df-44db-816a-fe21e480ad59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:03.974304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:03.974304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:03.974304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:03.974304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:03.974304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:03.974304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:03.974304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:03.974304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:03.974304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:03.974304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:03.974304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:03.974304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:54:03.974304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:54:03.974304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:54:03.975317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:54:03.975317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:03.975317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:03.975317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:03.975317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:03.975317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:03.975317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:03.975317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:03.975317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:03.975317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:03.975317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:03.975317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:03.975317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:03.975317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:03.975317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:03.975317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:03.975317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:03.976239 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:54:03.976239 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:54:03.976239 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:54:03.976239 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:54:03.976239 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:54:03.976239 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:54:03.976239 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:03.976239 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:03.976239 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 350/714] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:03.990836 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7729d8f8-e2df-44db-816a-fe21e480ad59 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:04.141970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7729d8f8-e2df-44db-816a-fe21e480ad59 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:04.153925 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516c869-c3ea-4337-9218-efef3ad0a7ba None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:04.177179 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18239341-eec1-4777-a6a9-319723260f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:04.177843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18239341-eec1-4777-a6a9-319723260f1b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:04.178120 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18239341-eec1-4777-a6a9-319723260f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:04.178390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18239341-eec1-4777-a6a9-319723260f1b None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:04.179172 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18239341-eec1-4777-a6a9-319723260f1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:04.179474 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18239341-eec1-4777-a6a9-319723260f1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:04.181035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18239341-eec1-4777-a6a9-319723260f1b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:04.196753 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:04.197840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 351/715] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:54:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:04.199250 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18239341-eec1-4777-a6a9-319723260f1b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:04.225410 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbaa938-d404-41ce-95ab-4f80bcc0792a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:04.226062 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbaa938-d404-41ce-95ab-4f80bcc0792a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:04.226341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbaa938-d404-41ce-95ab-4f80bcc0792a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:04.226610 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbaa938-d404-41ce-95ab-4f80bcc0792a None admin] PATH_INFO: `/v3/services/network` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:04.227372 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbaa938-d404-41ce-95ab-4f80bcc0792a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:04.228833 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbaa938-d404-41ce-95ab-4f80bcc0792a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:04.258078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-9cbaa938-d404-41ce-95ab-4f80bcc0792a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 25 15:54:04.258078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:04.258078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:04.258078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:04.258078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:04.258078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:04.258078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:04.258078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:04.258078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 25 15:54:04.258078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:04.258078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:04.258078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:04.258078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:04.258078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:04.258078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:04.258078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:04.258078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:04.259489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:04.259489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:04.259489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:04.259489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:04.259489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:04.259489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:04.259489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:04.259489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:04.259489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:04.259489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:04.259489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:04.259489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:04.259489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:04.259489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:04.259489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:04.259489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:04.260362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:04.260362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:04.260362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:04.260362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:04.260362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:04.260362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:04.260362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:04.260362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:04.260362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:04.260362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:04.260362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:04.260362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:04.260362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 25 15:54:04.260362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:04.262523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 365/716] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:03 2022] GET /identity/v3/services/network => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:04.269098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbaa938-d404-41ce-95ab-4f80bcc0792a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:04.330867 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18239341-eec1-4777-a6a9-319723260f1b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:04.366806 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf1b94-4ad0-4fb9-b5ad-7ad8b12a2a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:04.367459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf1b94-4ad0-4fb9-b5ad-7ad8b12a2a41 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:04.367788 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf1b94-4ad0-4fb9-b5ad-7ad8b12a2a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:04.368063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf1b94-4ad0-4fb9-b5ad-7ad8b12a2a41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:04.369858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf1b94-4ad0-4fb9-b5ad-7ad8b12a2a41 None admin] RBAC: Adding query filter params (role.id=b1e13c449fa64b3e88b6832bb767214b, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=3622c5ccb7534e7c8e8342b4f6643bc3) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:04.370171 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf1b94-4ad0-4fb9-b5ad-7ad8b12a2a41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:04.371638 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf1b94-4ad0-4fb9-b5ad-7ad8b12a2a41 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:04.383038 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 352/717] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:54:04 2022] GET /identity/v3/role_assignments?user.id=3622c5ccb7534e7c8e8342b4f6643bc3&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:04.411479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:04.495738 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbaa938-d404-41ce-95ab-4f80bcc0792a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:04.546625 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ad37c-6727-44ee-9671-8cd07be1b2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:04.547319 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ad37c-6727-44ee-9671-8cd07be1b2bb None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:04.547607 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ad37c-6727-44ee-9671-8cd07be1b2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:04.547888 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ad37c-6727-44ee-9671-8cd07be1b2bb None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:04.548643 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ad37c-6727-44ee-9671-8cd07be1b2bb None admin] RBAC: Adding query filter params (name=network) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:04.548953 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ad37c-6727-44ee-9671-8cd07be1b2bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:04.550364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ad37c-6727-44ee-9671-8cd07be1b2bb None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:04.562086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 366/718] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Jun 25 15:54:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:04.568085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf1b94-4ad0-4fb9-b5ad-7ad8b12a2a41 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:04.593769 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244aa546-8084-4e46-943e-bc8776f48fc1 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:04.594099 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244aa546-8084-4e46-943e-bc8776f48fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:04.594337 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244aa546-8084-4e46-943e-bc8776f48fc1 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:04.595838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 367/719] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:04.601649 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d4cd7-acd7-4634-a080-f266fd2ba60c None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:04.602327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d4cd7-acd7-4634-a080-f266fd2ba60c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:04.602955 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800d4cd7-acd7-4634-a080-f266fd2ba60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:04.623194 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-800d4cd7-acd7-4634-a080-f266fd2ba60c None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:04.624821 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-800d4cd7-acd7-4634-a080-f266fd2ba60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:04.636048 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:04.689743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf1b94-4ad0-4fb9-b5ad-7ad8b12a2a41 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:04.702644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 368/720] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:54:04.727185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d7c5c-0688-4f98-a7d5-aab66f9147cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:04.728121 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d7c5c-0688-4f98-a7d5-aab66f9147cc None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:04.728346 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d7c5c-0688-4f98-a7d5-aab66f9147cc None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:04.728634 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d7c5c-0688-4f98-a7d5-aab66f9147cc None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:04.729346 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d7c5c-0688-4f98-a7d5-aab66f9147cc None admin] RBAC: Adding query filter params (type=network) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:04.729646 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d7c5c-0688-4f98-a7d5-aab66f9147cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:04.730222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d4cd7-acd7-4634-a080-f266fd2ba60c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:04.731022 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d7c5c-0688-4f98-a7d5-aab66f9147cc None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:04.740500 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 353/721] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Jun 25 15:54:04 2022] GET /identity/v3/services?type=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:04.811740 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:04.907581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800d4cd7-acd7-4634-a080-f266fd2ba60c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:04.925415 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c45d3-16f7-4ec4-bf9f-b8dbf2036334 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:04.925702 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c45d3-16f7-4ec4-bf9f-b8dbf2036334 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:04.925980 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c45d3-16f7-4ec4-bf9f-b8dbf2036334 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:04.927580 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 354/722] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:54:04.932862 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1162e-a754-4a38-ba69-9f9a69a24f3c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:04.933269 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1162e-a754-4a38-ba69-9f9a69a24f3c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:04.933640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1162e-a754-4a38-ba69-9f9a69a24f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:04.954328 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-3fa1162e-a754-4a38-ba69-9f9a69a24f3c None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:04.955747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3fa1162e-a754-4a38-ba69-9f9a69a24f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:04.956127 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ea914-bb75-437c-a613-bb141c15ba7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:04.957002 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ea914-bb75-437c-a613-bb141c15ba7c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:04.957291 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ea914-bb75-437c-a613-bb141c15ba7c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:04.957568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ea914-bb75-437c-a613-bb141c15ba7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:04.974258 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ea914-bb75-437c-a613-bb141c15ba7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:54:04.983507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-fd2ea914-bb75-437c-a613-bb141c15ba7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:54:04.983507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:04.983507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:04.983507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:04.983507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:04.983507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:04.983507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:04.983507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:04.983507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:04.983507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:04.983507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:04.983507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:04.983507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:54:04.983507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:54:04.983507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:54:04.983507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:54:04.984564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:54:04.984564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:54:04.984564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:54:04.984564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:54:04.984564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:04.984564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:04.984564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:04.984564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:04.984564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:04.984564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:04.984564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:04.984564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:04.984564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:04.984564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:04.984564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:04.984564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:04.985426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:04.985426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:04.985426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:04.985426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:04.985426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:54:04.985426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:54:04.985426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:54:04.985426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:54:04.985426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:54:04.985426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:54:04.985426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:54:04.985426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:04.986587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 369/723] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:54:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:04.993501 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ea914-bb75-437c-a613-bb141c15ba7c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:05.031376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 355/724] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 25 15:54:05.069571 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1162e-a754-4a38-ba69-9f9a69a24f3c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:05.124848 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:05.143147 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ea914-bb75-437c-a613-bb141c15ba7c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:05.152760 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:05.194094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a01677-d858-40f4-a9cf-9dbe8e11131b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:05.194771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a01677-d858-40f4-a9cf-9dbe8e11131b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:05.195049 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a01677-d858-40f4-a9cf-9dbe8e11131b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:05.195321 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a01677-d858-40f4-a9cf-9dbe8e11131b None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:05.196327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a01677-d858-40f4-a9cf-9dbe8e11131b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:05.196661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a01677-d858-40f4-a9cf-9dbe8e11131b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:05.198037 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a01677-d858-40f4-a9cf-9dbe8e11131b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:05.218526 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 370/725] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:54:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:05.221407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1162e-a754-4a38-ba69-9f9a69a24f3c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:05.223815 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a01677-d858-40f4-a9cf-9dbe8e11131b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:05.272715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab5c86-27f4-4ec4-a6c7-f6901a9377ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:05.273516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab5c86-27f4-4ec4-a6c7-f6901a9377ec None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:05.273903 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab5c86-27f4-4ec4-a6c7-f6901a9377ec None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:05.274156 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab5c86-27f4-4ec4-a6c7-f6901a9377ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:05.288432 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab5c86-27f4-4ec4-a6c7-f6901a9377ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:54:05.293148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-44ab5c86-27f4-4ec4-a6c7-f6901a9377ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:54:05.293148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:05.293148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:05.293148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:05.293148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:05.293148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:05.293148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:05.293148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:05.293148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:05.293148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:05.293148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:05.293148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:05.293148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:54:05.293148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:54:05.293148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:54:05.293148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:54:05.294087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:54:05.294087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:54:05.294087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:54:05.294087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:54:05.294087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:05.294087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:05.294087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:05.294087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:05.294087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:05.294087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:05.294087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:05.294087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:05.294087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:05.294087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:05.294087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:05.294087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:05.294934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:05.294934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:05.294934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:05.294934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:05.294934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:54:05.294934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:54:05.294934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:54:05.294934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:54:05.294934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:54:05.294934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:54:05.294934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:54:05.294934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:05.294934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 356/726] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:54:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:05.300537 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab5c86-27f4-4ec4-a6c7-f6901a9377ec None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:05.432097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a01677-d858-40f4-a9cf-9dbe8e11131b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:05.444866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab5c86-27f4-4ec4-a6c7-f6901a9377ec None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:05.468612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:05.492568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75111c90-440f-4d28-a8f3-b219ade56c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:05.493683 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75111c90-440f-4d28-a8f3-b219ade56c84 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:05.494176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75111c90-440f-4d28-a8f3-b219ade56c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:05.494639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75111c90-440f-4d28-a8f3-b219ade56c84 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:05.495584 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75111c90-440f-4d28-a8f3-b219ade56c84 None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:05.496141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75111c90-440f-4d28-a8f3-b219ade56c84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:05.497830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75111c90-440f-4d28-a8f3-b219ade56c84 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:05.506447 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 357/727] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:54:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:05.512461 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0813e-49b7-4a60-8aa0-ea49f81a5f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:05.512461 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0813e-49b7-4a60-8aa0-ea49f81a5f82 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:05.512461 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0813e-49b7-4a60-8aa0-ea49f81a5f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:05.512461 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0813e-49b7-4a60-8aa0-ea49f81a5f82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:05.513790 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75111c90-440f-4d28-a8f3-b219ade56c84 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:05.541503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0813e-49b7-4a60-8aa0-ea49f81a5f82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:05.542970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0813e-49b7-4a60-8aa0-ea49f81a5f82 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:05.572231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-51d0813e-49b7-4a60-8aa0-ea49f81a5f82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:05.572231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:05.572231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:05.572231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:05.572231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:05.572231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:05.572231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:05.572231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:05.572231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:05.572231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:05.572231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:05.572231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:05.572231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:05.572231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:05.572231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:05.573361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:05.573361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:05.573361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:05.573361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:05.573361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:05.573361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:05.573361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:05.573361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:05.573361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:05.573361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:05.573361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:05.573361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:05.573361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:05.573361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:05.573361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:05.573361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:05.573361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:05.574480 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:05.574480 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:05.574480 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:05.574480 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:05.581284 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 371/728] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:05.586982 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0813e-49b7-4a60-8aa0-ea49f81a5f82 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:05.619848 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:05.698156 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75111c90-440f-4d28-a8f3-b219ade56c84 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:05.771878 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0813e-49b7-4a60-8aa0-ea49f81a5f82 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:05.774663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9985ea8-ec18-4ac2-908a-8d3116f6b895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:05.775697 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9985ea8-ec18-4ac2-908a-8d3116f6b895 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:05.776152 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9985ea8-ec18-4ac2-908a-8d3116f6b895 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:05.776621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9985ea8-ec18-4ac2-908a-8d3116f6b895 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:05.799804 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9985ea8-ec18-4ac2-908a-8d3116f6b895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:05.803271 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9985ea8-ec18-4ac2-908a-8d3116f6b895 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:05.836665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-d9985ea8-ec18-4ac2-908a-8d3116f6b895 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:05.836665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:05.836665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:05.836665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:05.836665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:05.836665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:05.836665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:05.836665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:05.836665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:05.836665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:05.836665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:05.836665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:05.836665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:05.836665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:05.836665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:05.837615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:05.837615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:05.837615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:05.837615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:05.837615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:05.837615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:05.837615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:05.837615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:05.837615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:05.837615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:05.837615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:05.837615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:05.837615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:05.837615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:05.837615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:05.837615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:05.837615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:05.838536 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:05.838536 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:05.838536 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:05.838536 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:05.847552 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacca8f3-b21a-4174-879b-c5d35bd0a33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:05.852726 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 358/729] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:05.853001 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacca8f3-b21a-4174-879b-c5d35bd0a33c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:05.853596 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacca8f3-b21a-4174-879b-c5d35bd0a33c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:05.854055 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacca8f3-b21a-4174-879b-c5d35bd0a33c None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:05.854973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacca8f3-b21a-4174-879b-c5d35bd0a33c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:05.855455 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacca8f3-b21a-4174-879b-c5d35bd0a33c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:05.857058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacca8f3-b21a-4174-879b-c5d35bd0a33c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:05.862002 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9985ea8-ec18-4ac2-908a-8d3116f6b895 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:05.889508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 372/730] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:05.893054 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacca8f3-b21a-4174-879b-c5d35bd0a33c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:05.920749 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:06.052866 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:06.080836 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9985ea8-ec18-4ac2-908a-8d3116f6b895 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:06.096573 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacca8f3-b21a-4174-879b-c5d35bd0a33c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:06.153043 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7641a7-7820-49b5-9f59-a775f0f2983e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:06.153719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7641a7-7820-49b5-9f59-a775f0f2983e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:06.153999 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7641a7-7820-49b5-9f59-a775f0f2983e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:06.154274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7641a7-7820-49b5-9f59-a775f0f2983e None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:06.155006 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7641a7-7820-49b5-9f59-a775f0f2983e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:06.155308 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7641a7-7820-49b5-9f59-a775f0f2983e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:06.156723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7641a7-7820-49b5-9f59-a775f0f2983e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:06.167041 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5e573-5d4f-4de1-bdff-ab717e4e0460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:06.167843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5e573-5d4f-4de1-bdff-ab717e4e0460 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:06.168124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5e573-5d4f-4de1-bdff-ab717e4e0460 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:06.168394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5e573-5d4f-4de1-bdff-ab717e4e0460 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:06.186468 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5e573-5d4f-4de1-bdff-ab717e4e0460 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:06.187928 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5e573-5d4f-4de1-bdff-ab717e4e0460 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:06.192794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 359/731] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:06.193747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7641a7-7820-49b5-9f59-a775f0f2983e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:06.226799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-b4e5e573-5d4f-4de1-bdff-ab717e4e0460 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 25 15:54:06.226799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:06.226799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:06.226799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:06.226799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:06.226799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:06.226799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:06.226799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:06.226799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:06.226799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:06.226799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:06.226799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:06.226799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:54:06.226799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:54:06.226799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:54:06.226799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:54:06.227843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:06.227843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:06.227843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:54:06.227843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:06.227843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:54:06.227843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:06.227843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:06.227843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:06.227843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:06.227843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:06.227843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:06.227843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:06.227843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:06.227843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:06.227843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:06.227843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:06.227843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:06.227843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:06.228808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:06.228808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:06.228808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:54:06.228808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:54:06.228808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:54:06.228808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:54:06.228808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:54:06.228808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:54:06.228808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 25 15:54:06.228808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:06.233014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 373/732] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:54:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:06.233865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5e573-5d4f-4de1-bdff-ab717e4e0460 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:06.313063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:06.325606 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7641a7-7820-49b5-9f59-a775f0f2983e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:06.385515 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442262a5-e5f2-4872-8641-b28c62eeefd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:06.386339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442262a5-e5f2-4872-8641-b28c62eeefd4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:06.386682 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442262a5-e5f2-4872-8641-b28c62eeefd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:06.386948 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442262a5-e5f2-4872-8641-b28c62eeefd4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:06.394758 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442262a5-e5f2-4872-8641-b28c62eeefd4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:06.396209 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442262a5-e5f2-4872-8641-b28c62eeefd4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:06.411426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-442262a5-e5f2-4872-8641-b28c62eeefd4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 25 15:54:06.411426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:06.411426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:06.411426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:06.411426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:06.411426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:06.411426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:06.411426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:06.411426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:06.411426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:06.411426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:06.411426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:06.411426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:54:06.411426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:54:06.411426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:54:06.411426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:54:06.412453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:06.412453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:06.412453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:54:06.412453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:06.412453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:54:06.412453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:06.412453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:06.412453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:06.412453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:06.412453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:06.412453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:06.412453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:06.412453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:06.412453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:06.412453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:06.412453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:06.412453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:06.412453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:06.413480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:06.413480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:06.413480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:54:06.413480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:54:06.413480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:54:06.413480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:54:06.413480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:54:06.413480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:54:06.413480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 25 15:54:06.413480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:06.413480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 360/733] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Jun 25 15:54:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:06.414617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97827d-2980-4fce-9283-63ba34481f24 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:06.414617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97827d-2980-4fce-9283-63ba34481f24 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:06.414855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97827d-2980-4fce-9283-63ba34481f24 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:06.424436 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 361/734] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:06.424436 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97827d-2980-4fce-9283-63ba34481f24 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:06.439370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5e573-5d4f-4de1-bdff-ab717e4e0460 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:06.475449 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442603a-8808-49ae-9bfc-b1cb429e10a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:06.476571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442603a-8808-49ae-9bfc-b1cb429e10a7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:06.477284 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442603a-8808-49ae-9bfc-b1cb429e10a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:06.478016 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442603a-8808-49ae-9bfc-b1cb429e10a7 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:06.479055 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442603a-8808-49ae-9bfc-b1cb429e10a7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:06.479556 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442603a-8808-49ae-9bfc-b1cb429e10a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:06.481391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442603a-8808-49ae-9bfc-b1cb429e10a7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:06.496806 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:06.500995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 374/735] 10.222.0.13 () {56 vars in 1135 bytes} [Sat Jun 25 15:54:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:06.502731 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07d456-5a62-49cc-8e24-e63e1439cdc8 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:06.503215 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07d456-5a62-49cc-8e24-e63e1439cdc8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:06.503701 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07d456-5a62-49cc-8e24-e63e1439cdc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:06.527458 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5d07d456-5a62-49cc-8e24-e63e1439cdc8 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:06.529079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5d07d456-5a62-49cc-8e24-e63e1439cdc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:06.534391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97827d-2980-4fce-9283-63ba34481f24 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:06.574132 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4937a9f-1aef-4f7a-a440-57d65b29ce3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:06.575152 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4937a9f-1aef-4f7a-a440-57d65b29ce3a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:06.575642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4937a9f-1aef-4f7a-a440-57d65b29ce3a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:06.576147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4937a9f-1aef-4f7a-a440-57d65b29ce3a None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:06.577259 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4937a9f-1aef-4f7a-a440-57d65b29ce3a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:06.577758 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4937a9f-1aef-4f7a-a440-57d65b29ce3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:06.579594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4937a9f-1aef-4f7a-a440-57d65b29ce3a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:06.601017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 362/736] 10.222.0.13 () {56 vars in 1149 bytes} [Sat Jun 25 15:54:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:06.602545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4937a9f-1aef-4f7a-a440-57d65b29ce3a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:06.607694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:06.612195 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 375/737] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:54:06.613202 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07d456-5a62-49cc-8e24-e63e1439cdc8 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:06.714786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4937a9f-1aef-4f7a-a440-57d65b29ce3a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:06.740056 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07d456-5a62-49cc-8e24-e63e1439cdc8 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:06.750829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0f35e-e1dc-4d2d-ac35-323d2061fe7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:06.751658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0f35e-e1dc-4d2d-ac35-323d2061fe7b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:06.752075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0f35e-e1dc-4d2d-ac35-323d2061fe7b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:06.752545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0f35e-e1dc-4d2d-ac35-323d2061fe7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:06.765340 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:06.775436 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b227be-c61a-45b0-962d-1c8cbdf8a1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:06.776214 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b227be-c61a-45b0-962d-1c8cbdf8a1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:06.776384 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b227be-c61a-45b0-962d-1c8cbdf8a1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:06.776710 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b227be-c61a-45b0-962d-1c8cbdf8a1e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:06.825489 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0f35e-e1dc-4d2d-ac35-323d2061fe7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:06.826882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0f35e-e1dc-4d2d-ac35-323d2061fe7b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:06.829544 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:06.850259 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b227be-c61a-45b0-962d-1c8cbdf8a1e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:06.851735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b227be-c61a-45b0-962d-1c8cbdf8a1e9 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:06.941520 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 363/738] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:54:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:54:06.942692 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0f35e-e1dc-4d2d-ac35-323d2061fe7b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:06.959005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 376/739] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:54:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 25 15:54:06.960617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e541e1-07ce-4a0b-8fd5-da5ee9e7018d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:06.961082 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e541e1-07ce-4a0b-8fd5-da5ee9e7018d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:06.961645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e541e1-07ce-4a0b-8fd5-da5ee9e7018d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:06.967047 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 377/740] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:06.968070 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e541e1-07ce-4a0b-8fd5-da5ee9e7018d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:06.995808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:07.047543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0f35e-e1dc-4d2d-ac35-323d2061fe7b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:07.049826 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:07.069846 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e541e1-07ce-4a0b-8fd5-da5ee9e7018d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:07.082964 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14479c-8cc2-4473-9153-595e7c9c15ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:07.083542 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14479c-8cc2-4473-9153-595e7c9c15ae None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:07.083814 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14479c-8cc2-4473-9153-595e7c9c15ae None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:07.084086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14479c-8cc2-4473-9153-595e7c9c15ae None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:07.084844 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d14479c-8cc2-4473-9153-595e7c9c15ae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:07.086729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d14479c-8cc2-4473-9153-595e7c9c15ae None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:07.102165 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 364/741] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Jun 25 15:54:06 2022] POST /identity/v3/services => generated 251 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:54:07.103029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14479c-8cc2-4473-9153-595e7c9c15ae None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:07.126823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87be47-09a7-40b3-b602-f6d3a68a3501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:07.127638 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87be47-09a7-40b3-b602-f6d3a68a3501 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:07.127948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87be47-09a7-40b3-b602-f6d3a68a3501 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:07.128192 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87be47-09a7-40b3-b602-f6d3a68a3501 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:07.140312 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87be47-09a7-40b3-b602-f6d3a68a3501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:07.141920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87be47-09a7-40b3-b602-f6d3a68a3501 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:07.164203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-2f87be47-09a7-40b3-b602-f6d3a68a3501 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:07.164203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:07.164203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:07.164203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:07.164203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:07.164203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:07.164203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:07.164203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:07.164203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:07.164203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:07.164203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:07.164203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:07.164203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:07.164203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:07.164203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:07.165299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:07.165299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:07.165299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:07.165299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:07.165299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:07.165299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:07.165299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:07.165299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:07.165299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:07.165299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:07.165299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:07.165299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:07.165299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:07.165299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:07.165299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:07.165299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:07.165299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:07.166192 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:07.166192 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:07.166192 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:07.166192 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:07.166421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 378/742] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:07.167647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030a0d8-4b49-4796-abb7-30fa86a068d7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:07.167937 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030a0d8-4b49-4796-abb7-30fa86a068d7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:07.168218 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030a0d8-4b49-4796-abb7-30fa86a068d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:07.187243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-0030a0d8-4b49-4796-abb7-30fa86a068d7 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:07.188490 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0030a0d8-4b49-4796-abb7-30fa86a068d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:07.232176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14479c-8cc2-4473-9153-595e7c9c15ae None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:07.266886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 379/743] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:07 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:54:07.272456 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df042df-9341-48f3-861b-528dc974bef2 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:07.272456 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df042df-9341-48f3-861b-528dc974bef2 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:07.272456 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df042df-9341-48f3-861b-528dc974bef2 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:07.272456 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 380/744] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:07.272456 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df042df-9341-48f3-861b-528dc974bef2 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:07.275825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:07.275958 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2001e-7d7e-453b-b478-b5b2bd42c302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:07.277770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2001e-7d7e-453b-b478-b5b2bd42c302 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:07.277770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2001e-7d7e-453b-b478-b5b2bd42c302 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:07.277770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2001e-7d7e-453b-b478-b5b2bd42c302 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:07.290387 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b2001e-7d7e-453b-b478-b5b2bd42c302 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:07.291858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b2001e-7d7e-453b-b478-b5b2bd42c302 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:07.310477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-40b2001e-7d7e-453b-b478-b5b2bd42c302 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:07.310477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:07.310477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:07.310477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:07.310477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:07.310477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:07.310477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:07.310477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:07.310477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:07.310477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:07.310477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:07.310477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:07.310477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:07.310477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:07.310477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:07.311565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:07.311565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:07.311565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:07.311565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:07.311565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:07.311565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:07.311565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:07.311565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:07.311565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:07.311565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:07.311565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:07.311565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:07.311565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:07.311565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:07.311565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:07.311565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:07.311565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:07.312519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:07.312519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:07.312519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:07.312519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:07.312519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 365/745] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:07.313220 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886313b8-e0ba-454c-a525-f3c93b285704 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:07.313381 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886313b8-e0ba-454c-a525-f3c93b285704 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:07.313677 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886313b8-e0ba-454c-a525-f3c93b285704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:07.330011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-886313b8-e0ba-454c-a525-f3c93b285704 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:07.331203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-886313b8-e0ba-454c-a525-f3c93b285704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:07.343893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:07.386877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df042df-9341-48f3-861b-528dc974bef2 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:07.422336 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013eff0-852e-43bf-bca2-1846375e64ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:07.422999 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013eff0-852e-43bf-bca2-1846375e64ca None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:07.423280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013eff0-852e-43bf-bca2-1846375e64ca None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:07.423551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013eff0-852e-43bf-bca2-1846375e64ca None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:07.432781 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 366/746] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:07 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 25 15:54:07.432946 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013eff0-852e-43bf-bca2-1846375e64ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:07.432946 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013eff0-852e-43bf-bca2-1846375e64ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:07.433097 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886313b8-e0ba-454c-a525-f3c93b285704 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:07.433170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013eff0-852e-43bf-bca2-1846375e64ca None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:07.441980 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 381/747] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:07.443014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013eff0-852e-43bf-bca2-1846375e64ca None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:07.509941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:07.539940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886313b8-e0ba-454c-a525-f3c93b285704 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:07.552194 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013eff0-852e-43bf-bca2-1846375e64ca None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:07.581228 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63429bb2-8533-48ff-80fb-b3e6e01e2a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:07.582024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63429bb2-8533-48ff-80fb-b3e6e01e2a0b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:07.582327 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63429bb2-8533-48ff-80fb-b3e6e01e2a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:07.582876 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63429bb2-8533-48ff-80fb-b3e6e01e2a0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:07.588678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3e750-5dc1-4a68-b6e9-3786bc37c31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:07.589281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3e750-5dc1-4a68-b6e9-3786bc37c31f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:07.589583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3e750-5dc1-4a68-b6e9-3786bc37c31f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:07.589848 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3e750-5dc1-4a68-b6e9-3786bc37c31f None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:07.590581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3e750-5dc1-4a68-b6e9-3786bc37c31f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:07.590894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3e750-5dc1-4a68-b6e9-3786bc37c31f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:07.592320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3e750-5dc1-4a68-b6e9-3786bc37c31f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:07.595020 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63429bb2-8533-48ff-80fb-b3e6e01e2a0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:54:07.602594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-63429bb2-8533-48ff-80fb-b3e6e01e2a0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:54:07.602594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:07.602594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:07.602594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:07.602594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:07.602594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:07.602594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:07.602594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:07.602594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:07.602594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:07.602594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:07.602594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:07.602594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:54:07.602594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:54:07.602594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:54:07.602594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:54:07.603566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:54:07.603566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:54:07.603566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:54:07.603566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:54:07.603566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:07.603566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:07.603566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:07.603566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:07.603566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:07.603566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:07.603566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:07.603566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:07.603566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:07.603566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:07.603566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:07.603566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:07.604484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:07.604484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:07.604484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:07.604484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:07.604484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:54:07.604484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:54:07.604484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:54:07.604484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:54:07.604484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:54:07.604484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:54:07.604484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:54:07.604484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:07.604484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 367/748] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:54:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:07.605182 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63429bb2-8533-48ff-80fb-b3e6e01e2a0b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:07.610145 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 382/749] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:07.611199 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3e750-5dc1-4a68-b6e9-3786bc37c31f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:07.614416 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:07.713468 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63429bb2-8533-48ff-80fb-b3e6e01e2a0b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:07.717871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3e750-5dc1-4a68-b6e9-3786bc37c31f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:07.742532 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:07.750519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2858e53-8043-48e2-93e7-ef4719c191f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:07.751310 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2858e53-8043-48e2-93e7-ef4719c191f0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:07.751589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2858e53-8043-48e2-93e7-ef4719c191f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:07.751863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2858e53-8043-48e2-93e7-ef4719c191f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:07.755888 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfb7e1-4f1f-472d-9757-fc9fd3c40832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:07.756791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfb7e1-4f1f-472d-9757-fc9fd3c40832 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:07.757250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfb7e1-4f1f-472d-9757-fc9fd3c40832 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:07.757713 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfb7e1-4f1f-472d-9757-fc9fd3c40832 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:07.758887 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2858e53-8043-48e2-93e7-ef4719c191f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:07.759075 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cfb7e1-4f1f-472d-9757-fc9fd3c40832 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:07.760347 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2858e53-8043-48e2-93e7-ef4719c191f0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:07.760768 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cfb7e1-4f1f-472d-9757-fc9fd3c40832 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:07.776044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-a2858e53-8043-48e2-93e7-ef4719c191f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:07.776044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:07.776044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:07.776044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:07.776044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:07.776044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:07.776044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:07.776044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:07.776044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:07.776044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:07.776044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:07.776044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:07.776044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:54:07.776044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:54:07.776044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:54:07.777035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:54:07.777035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:07.777035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:07.777035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:07.777035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:07.777035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:07.777035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:07.777035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:07.777035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:07.777035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:07.777035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:07.777035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:07.777035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:07.777035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:07.777035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:07.777035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:07.777035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:07.777970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:54:07.777970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:54:07.777970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:54:07.777970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:54:07.777970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:54:07.777970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:54:07.777970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:07.777970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:07.778400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 383/750] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Jun 25 15:54:07 2022] POST /identity/v3/services => generated 240 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:54:07.778400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfb7e1-4f1f-472d-9757-fc9fd3c40832 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:07.782061 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 368/751] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:07.782887 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2858e53-8043-48e2-93e7-ef4719c191f0 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:07.884638 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:07.936213 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2858e53-8043-48e2-93e7-ef4719c191f0 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:07.937678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfb7e1-4f1f-472d-9757-fc9fd3c40832 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:07.977773 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364622e-9d0b-4347-ba46-294fd9df3059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:07.978582 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364622e-9d0b-4347-ba46-294fd9df3059 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:07.978872 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364622e-9d0b-4347-ba46-294fd9df3059 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:07.979132 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364622e-9d0b-4347-ba46-294fd9df3059 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:07.981384 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96a373-c573-47ff-9994-b8d4f479276a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:07.982544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96a373-c573-47ff-9994-b8d4f479276a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:07.983010 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96a373-c573-47ff-9994-b8d4f479276a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:07.983461 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96a373-c573-47ff-9994-b8d4f479276a None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:07.984438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96a373-c573-47ff-9994-b8d4f479276a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:07.984922 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96a373-c573-47ff-9994-b8d4f479276a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:07.986219 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364622e-9d0b-4347-ba46-294fd9df3059 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:07.987018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96a373-c573-47ff-9994-b8d4f479276a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:07.987690 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364622e-9d0b-4347-ba46-294fd9df3059 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:07.999304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 384/752] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:54:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:08.000540 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96a373-c573-47ff-9994-b8d4f479276a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:08.005590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-5364622e-9d0b-4347-ba46-294fd9df3059 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:08.005590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:08.005590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:08.005590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:08.005590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:08.005590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:08.005590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:08.005590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:08.005590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:08.005590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:08.005590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:08.005590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:08.005590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:54:08.005590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:54:08.005590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:54:08.006491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:54:08.006491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:08.006491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:08.006491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:08.006491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:08.006491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:08.006491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:08.006491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:08.006491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:08.006491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:08.006491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:08.006491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:08.006491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:08.006491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:08.006491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:08.006491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:08.006491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:08.007376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:54:08.007376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:54:08.007376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:54:08.007376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:54:08.007376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:54:08.007376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:54:08.007376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:08.007376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:08.007376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 369/753] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:08.007869 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364622e-9d0b-4347-ba46-294fd9df3059 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:08.064341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:08.122232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364622e-9d0b-4347-ba46-294fd9df3059 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:08.135547 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96a373-c573-47ff-9994-b8d4f479276a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:08.151345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:08.167099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee82b1a-8b6e-4ad7-8a5c-30850c0c6094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:08.168020 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee82b1a-8b6e-4ad7-8a5c-30850c0c6094 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:08.168433 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee82b1a-8b6e-4ad7-8a5c-30850c0c6094 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:08.168785 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee82b1a-8b6e-4ad7-8a5c-30850c0c6094 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:08.178683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995dd32-6f0e-43d0-acea-860592fd40ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:08.179843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995dd32-6f0e-43d0-acea-860592fd40ef None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:08.180161 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995dd32-6f0e-43d0-acea-860592fd40ef None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:08.180469 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995dd32-6f0e-43d0-acea-860592fd40ef None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:08.181277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995dd32-6f0e-43d0-acea-860592fd40ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:08.181864 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995dd32-6f0e-43d0-acea-860592fd40ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:08.182503 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee82b1a-8b6e-4ad7-8a5c-30850c0c6094 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:08.183335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995dd32-6f0e-43d0-acea-860592fd40ef None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:08.184151 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee82b1a-8b6e-4ad7-8a5c-30850c0c6094 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:08.198253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 385/754] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:54:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:08.199133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995dd32-6f0e-43d0-acea-860592fd40ef None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:08.205816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-dee82b1a-8b6e-4ad7-8a5c-30850c0c6094 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:08.205816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:08.205816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:08.205816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:08.205816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:08.205816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:08.205816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:08.205816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:08.205816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:08.205816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:08.205816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:08.205816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:08.205816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:08.205816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:08.205816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:08.206720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:08.206720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:08.206720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:08.206720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:08.206720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:08.206720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:08.206720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:08.206720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:08.206720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:08.206720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:08.206720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:08.206720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:08.206720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:08.206720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:08.206720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:08.206720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:08.206720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:08.207618 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:08.207618 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:08.207618 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:08.207618 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:08.207618 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 370/755] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:08.209401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee82b1a-8b6e-4ad7-8a5c-30850c0c6094 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:08.318632 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995dd32-6f0e-43d0-acea-860592fd40ef None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:08.320554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:08.339775 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee82b1a-8b6e-4ad7-8a5c-30850c0c6094 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:08.358844 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998075d0-39f9-4700-a136-1fc22c34d9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:08.359515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998075d0-39f9-4700-a136-1fc22c34d9ab None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:08.359802 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998075d0-39f9-4700-a136-1fc22c34d9ab None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:08.360086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998075d0-39f9-4700-a136-1fc22c34d9ab None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:08.360929 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998075d0-39f9-4700-a136-1fc22c34d9ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:08.361242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998075d0-39f9-4700-a136-1fc22c34d9ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:08.362689 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998075d0-39f9-4700-a136-1fc22c34d9ab None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:08.375059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 386/756] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:54:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:08.375922 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998075d0-39f9-4700-a136-1fc22c34d9ab None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:08.377307 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d3db6-19d8-4343-b496-417297f332a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:08.378288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d3db6-19d8-4343-b496-417297f332a8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:08.378707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d3db6-19d8-4343-b496-417297f332a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:08.379219 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d3db6-19d8-4343-b496-417297f332a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:08.380303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d3db6-19d8-4343-b496-417297f332a8 None admin] RBAC: Adding query filter params (role.id=b1e13c449fa64b3e88b6832bb767214b, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=19788c4239ca44f9a31a7d32086c8947) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:08.380766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d3db6-19d8-4343-b496-417297f332a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:08.382357 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d3db6-19d8-4343-b496-417297f332a8 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:08.396783 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 371/757] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:54:08 2022] GET /identity/v3/role_assignments?user.id=19788c4239ca44f9a31a7d32086c8947&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:08.398126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d3db6-19d8-4343-b496-417297f332a8 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:08.478295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:08.481894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998075d0-39f9-4700-a136-1fc22c34d9ab None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:08.517630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec9748-387e-46f1-8933-0075f47befe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:08.518359 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec9748-387e-46f1-8933-0075f47befe2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:08.518644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec9748-387e-46f1-8933-0075f47befe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:08.518921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec9748-387e-46f1-8933-0075f47befe2 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:08.519660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec9748-387e-46f1-8933-0075f47befe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:08.519970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec9748-387e-46f1-8933-0075f47befe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:08.522756 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec9748-387e-46f1-8933-0075f47befe2 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:08.534975 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 387/758] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:08.544447 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec9748-387e-46f1-8933-0075f47befe2 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:08.592795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:08.597937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d3db6-19d8-4343-b496-417297f332a8 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:08.658631 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1343f3-4f73-4bd4-8512-5468f2f82d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:08.659220 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1343f3-4f73-4bd4-8512-5468f2f82d8c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:08.659504 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1343f3-4f73-4bd4-8512-5468f2f82d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:08.659783 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1343f3-4f73-4bd4-8512-5468f2f82d8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:08.660825 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1343f3-4f73-4bd4-8512-5468f2f82d8c None admin] RBAC: Adding query filter params (role.id=7f1996bcf7e54638b4f53b3d8ba83495, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=28b3e73439e64050b406a9f3e9b0160a) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:08.661140 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1343f3-4f73-4bd4-8512-5468f2f82d8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:08.662642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1343f3-4f73-4bd4-8512-5468f2f82d8c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:08.675619 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 372/759] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:54:08 2022] GET /identity/v3/role_assignments?user.id=28b3e73439e64050b406a9f3e9b0160a&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=7f1996bcf7e54638b4f53b3d8ba83495 => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:08.701390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec9748-387e-46f1-8933-0075f47befe2 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:08.751014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170498da-ab31-4312-9b54-0234c568ba43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:08.751823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170498da-ab31-4312-9b54-0234c568ba43 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:08.752104 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170498da-ab31-4312-9b54-0234c568ba43 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:08.752378 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170498da-ab31-4312-9b54-0234c568ba43 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:08.765084 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170498da-ab31-4312-9b54-0234c568ba43 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:08.766869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170498da-ab31-4312-9b54-0234c568ba43 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:08.782225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-170498da-ab31-4312-9b54-0234c568ba43 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 25 15:54:08.782225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:08.782225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:08.782225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:08.782225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:08.782225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:08.782225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:08.782225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:08.782225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:08.782225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:08.782225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:08.782225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:08.782225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:54:08.782225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:54:08.782225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:54:08.782225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:54:08.783629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:08.783629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:08.783629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:54:08.783629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:08.783629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:54:08.783629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:08.783629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:08.783629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:08.783629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:08.783629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:08.783629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:08.783629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:08.783629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:08.783629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:08.783629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:08.783629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:08.783629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:08.783629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:08.784801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:08.784801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:08.784801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:54:08.784801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:54:08.784801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:54:08.784801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:54:08.784801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:54:08.784801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:54:08.784801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 25 15:54:08.784801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:08.788732 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 388/760] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Jun 25 15:54:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:08.789367 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1343f3-4f73-4bd4-8512-5468f2f82d8c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:08.945920 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1343f3-4f73-4bd4-8512-5468f2f82d8c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:08.954376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:08.984422 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b129e1-f1a3-4f5f-bbd2-f95258400498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:08.985125 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b129e1-f1a3-4f5f-bbd2-f95258400498 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:08.985399 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b129e1-f1a3-4f5f-bbd2-f95258400498 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:08.985667 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b129e1-f1a3-4f5f-bbd2-f95258400498 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:08.986517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b129e1-f1a3-4f5f-bbd2-f95258400498 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:08.986816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b129e1-f1a3-4f5f-bbd2-f95258400498 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:08.988254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b129e1-f1a3-4f5f-bbd2-f95258400498 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:09.013803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 373/761] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Jun 25 15:54:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:09.019139 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170498da-ab31-4312-9b54-0234c568ba43 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:09.147790 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:09.176132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170498da-ab31-4312-9b54-0234c568ba43 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:09.221380 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da7ef3-fd93-4e7d-b0ae-205e648b1d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:09.222563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da7ef3-fd93-4e7d-b0ae-205e648b1d63 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:09.223061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da7ef3-fd93-4e7d-b0ae-205e648b1d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:09.223535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da7ef3-fd93-4e7d-b0ae-205e648b1d63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:09.238647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da7ef3-fd93-4e7d-b0ae-205e648b1d63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:09.240385 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da7ef3-fd93-4e7d-b0ae-205e648b1d63 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:09.265543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-e5da7ef3-fd93-4e7d-b0ae-205e648b1d63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:09.265543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:09.265543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:09.265543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:09.265543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:09.265543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:09.265543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:09.265543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:09.265543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:09.265543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:09.265543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:09.265543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:09.265543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:09.265543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:09.265543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:09.266832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:09.266832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:09.266832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:09.266832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:09.266832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:09.266832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:09.266832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:09.266832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:09.266832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:09.266832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:09.266832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:09.266832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:09.266832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:09.266832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:09.266832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:09.266832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:09.266832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:09.267766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:09.267766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:09.267766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:09.267766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:09.271418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 389/762] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:09.277282 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b129e1-f1a3-4f5f-bbd2-f95258400498 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:09.374812 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac9ac2-ffe6-4fac-97a2-9eae2bf57cd8 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:09.375185 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac9ac2-ffe6-4fac-97a2-9eae2bf57cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:09.375543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac9ac2-ffe6-4fac-97a2-9eae2bf57cd8 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:09.377158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 390/763] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:54:09.383785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f21f4-b6b6-4640-820d-88c72088a26b None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:09.384145 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f21f4-b6b6-4640-820d-88c72088a26b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:09.384339 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f21f4-b6b6-4640-820d-88c72088a26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:09.420642 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f65f21f4-b6b6-4640-820d-88c72088a26b None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:09.421854 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f65f21f4-b6b6-4640-820d-88c72088a26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:09.458534 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b129e1-f1a3-4f5f-bbd2-f95258400498 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:09.522704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718bf39-41f7-4f52-a715-bf0c9316ac35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:09.523285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718bf39-41f7-4f52-a715-bf0c9316ac35 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:09.523565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718bf39-41f7-4f52-a715-bf0c9316ac35 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:09.523959 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718bf39-41f7-4f52-a715-bf0c9316ac35 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:09.524898 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718bf39-41f7-4f52-a715-bf0c9316ac35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:09.525207 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718bf39-41f7-4f52-a715-bf0c9316ac35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:09.526592 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718bf39-41f7-4f52-a715-bf0c9316ac35 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:09.548328 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 374/764] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:09.550502 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 391/765] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:09.554261 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718bf39-41f7-4f52-a715-bf0c9316ac35 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:09.586675 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f21f4-b6b6-4640-820d-88c72088a26b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:09.618106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:09.707961 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f21f4-b6b6-4640-820d-88c72088a26b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:09.736772 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:09.753910 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060de7e-6043-4e67-ba83-4c47dfe0c47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:09.758347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060de7e-6043-4e67-ba83-4c47dfe0c47a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:09.758347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060de7e-6043-4e67-ba83-4c47dfe0c47a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:09.758347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060de7e-6043-4e67-ba83-4c47dfe0c47a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:09.758347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1060de7e-6043-4e67-ba83-4c47dfe0c47a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:09.758347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1060de7e-6043-4e67-ba83-4c47dfe0c47a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:09.781598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-1060de7e-6043-4e67-ba83-4c47dfe0c47a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 25 15:54:09.781598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:09.781598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:09.781598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:09.781598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:09.781598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:09.781598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:09.781598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:09.781598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 25 15:54:09.781598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:09.781598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:09.781598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:09.781598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:09.781598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:09.781598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:09.781598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:09.782995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:09.782995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:09.782995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:09.782995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:09.782995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:09.782995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:09.782995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:09.782995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:09.782995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:09.782995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:09.782995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:09.782995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:09.782995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:09.782995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:09.782995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:09.782995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:09.783816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:09.783816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:09.783816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:09.783816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:09.783816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:09.783816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:09.783816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:09.783816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:09.783816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:09.783816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:09.783816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:09.783816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:09.783816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:09.783816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 25 15:54:09.783816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:09.787309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 392/766] 10.222.0.13 () {56 vars in 1129 bytes} [Sat Jun 25 15:54:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:09.795990 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060de7e-6043-4e67-ba83-4c47dfe0c47a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:09.874385 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718bf39-41f7-4f52-a715-bf0c9316ac35 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:09.920546 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060de7e-6043-4e67-ba83-4c47dfe0c47a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:09.935934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b9b22-1a08-48ce-a159-cbc58153c797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:09.936918 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b9b22-1a08-48ce-a159-cbc58153c797 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:09.937298 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b9b22-1a08-48ce-a159-cbc58153c797 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:09.937657 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b9b22-1a08-48ce-a159-cbc58153c797 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:09.944745 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b9b22-1a08-48ce-a159-cbc58153c797 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:09.946376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b9b22-1a08-48ce-a159-cbc58153c797 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:09.949574 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:09.961431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-4e8b9b22-1a08-48ce-a159-cbc58153c797 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:09.961431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:09.961431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:09.961431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:09.961431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:09.961431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:09.961431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:09.961431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:09.961431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:09.961431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:09.961431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:09.961431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:09.961431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:54:09.961431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:54:09.961431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:54:09.962543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:54:09.962543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:09.962543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:09.962543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:09.962543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:09.962543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:09.962543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:09.962543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:09.962543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:09.962543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:09.962543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:09.962543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:09.962543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:09.962543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:09.962543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:09.962543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:09.962543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:09.963663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:54:09.963663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:54:09.963663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:54:09.963663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:54:09.963663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:54:09.963663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:54:09.963663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:09.963663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:09.965361 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 375/767] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:09.970099 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209dcbe-194d-4d48-9e0e-5e2957fda43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:09.970936 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b9b22-1a08-48ce-a159-cbc58153c797 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:09.971099 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209dcbe-194d-4d48-9e0e-5e2957fda43b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:09.971607 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209dcbe-194d-4d48-9e0e-5e2957fda43b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:09.972019 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209dcbe-194d-4d48-9e0e-5e2957fda43b None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:09.972895 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209dcbe-194d-4d48-9e0e-5e2957fda43b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:09.973337 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209dcbe-194d-4d48-9e0e-5e2957fda43b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:09.974960 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209dcbe-194d-4d48-9e0e-5e2957fda43b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:09.991279 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 393/768] 10.222.0.13 () {56 vars in 1124 bytes} [Sat Jun 25 15:54:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:10.000469 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209dcbe-194d-4d48-9e0e-5e2957fda43b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:10.101949 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:10.118261 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b9b22-1a08-48ce-a159-cbc58153c797 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:10.125848 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209dcbe-194d-4d48-9e0e-5e2957fda43b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:10.155866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae2a76-0823-4513-8e03-142dc37ab298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:10.156556 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae2a76-0823-4513-8e03-142dc37ab298 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:10.156845 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae2a76-0823-4513-8e03-142dc37ab298 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:10.157120 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae2a76-0823-4513-8e03-142dc37ab298 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:10.157910 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae2a76-0823-4513-8e03-142dc37ab298 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:10.158219 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae2a76-0823-4513-8e03-142dc37ab298 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:10.159658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae2a76-0823-4513-8e03-142dc37ab298 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:10.171961 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5ed55-5e39-4e86-bb01-9822d99d4bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:10.172141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 376/769] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:54:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:10.173143 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5ed55-5e39-4e86-bb01-9822d99d4bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:10.173618 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5ed55-5e39-4e86-bb01-9822d99d4bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:10.174098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5ed55-5e39-4e86-bb01-9822d99d4bb6 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:10.174998 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5ed55-5e39-4e86-bb01-9822d99d4bb6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:10.175492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5ed55-5e39-4e86-bb01-9822d99d4bb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:10.177382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5ed55-5e39-4e86-bb01-9822d99d4bb6 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:10.181332 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae2a76-0823-4513-8e03-142dc37ab298 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:10.185781 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 394/770] 10.222.0.13 () {56 vars in 1124 bytes} [Sat Jun 25 15:54:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:10.191621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5ed55-5e39-4e86-bb01-9822d99d4bb6 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:10.253509 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:10.313130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5ed55-5e39-4e86-bb01-9822d99d4bb6 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:10.321636 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae2a76-0823-4513-8e03-142dc37ab298 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:10.357057 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10355cb2-d06e-46bc-887c-d00359ced3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:10.358323 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10355cb2-d06e-46bc-887c-d00359ced3bb None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:10.359020 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10355cb2-d06e-46bc-887c-d00359ced3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:10.359510 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10355cb2-d06e-46bc-887c-d00359ced3bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:10.361170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10355cb2-d06e-46bc-887c-d00359ced3bb None admin] RBAC: Adding query filter params (interface=public, service_id=5c95b74d819f40fda493a8c4017fd96a, region_id=RegionOne) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:10.361948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10355cb2-d06e-46bc-887c-d00359ced3bb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:10.364320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10355cb2-d06e-46bc-887c-d00359ced3bb None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:10.367871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b6477-2b89-4469-b9db-290bc19390cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:10.368878 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b6477-2b89-4469-b9db-290bc19390cd None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:10.369228 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b6477-2b89-4469-b9db-290bc19390cd None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:10.369583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b6477-2b89-4469-b9db-290bc19390cd None admin] PATH_INFO: `/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/3622c5ccb7534e7c8e8342b4f6643bc3/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:10.371706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b6477-2b89-4469-b9db-290bc19390cd None admin] RBAC: Authorizing `identity:create_grant(project_id=3600648eb94b434b8c797d2f454e46b8, user_id=3622c5ccb7534e7c8e8342b4f6643bc3, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:10.373236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b6477-2b89-4469-b9db-290bc19390cd None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:10.380645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 395/771] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Jun 25 15:54:10 2022] GET /identity/v3/endpoints?service_id=5c95b74d819f40fda493a8c4017fd96a&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:10.380645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f269c7b-5879-4781-a94b-0ef858be8d2e None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:10.380645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f269c7b-5879-4781-a94b-0ef858be8d2e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:10.380645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f269c7b-5879-4781-a94b-0ef858be8d2e None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:10.382395 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 396/772] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:10.387183 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f269c7b-5879-4781-a94b-0ef858be8d2e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:10.406864 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 377/773] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:54:10 2022] PUT /identity/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/3622c5ccb7534e7c8e8342b4f6643bc3/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:54:10.410002 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420aae7-e0ed-49e2-832b-06be47bc6b4f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:10.411123 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420aae7-e0ed-49e2-832b-06be47bc6b4f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:10.411830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420aae7-e0ed-49e2-832b-06be47bc6b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:10.421380 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:10.438886 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-3420aae7-e0ed-49e2-832b-06be47bc6b4f None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:10.441249 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3420aae7-e0ed-49e2-832b-06be47bc6b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:10.519739 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 378/774] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:10.523374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f269c7b-5879-4781-a94b-0ef858be8d2e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:10.543431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420aae7-e0ed-49e2-832b-06be47bc6b4f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:10.547787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:10.570510 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59913eb3-fc96-4130-8bd5-ebf7095de810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:10.572232 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59913eb3-fc96-4130-8bd5-ebf7095de810 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:10.572730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59913eb3-fc96-4130-8bd5-ebf7095de810 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:10.573439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59913eb3-fc96-4130-8bd5-ebf7095de810 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:10.574739 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59913eb3-fc96-4130-8bd5-ebf7095de810 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:10.576811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59913eb3-fc96-4130-8bd5-ebf7095de810 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:10.586507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 397/775] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Jun 25 15:54:10 2022] GET /identity/v3/services => generated 772 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:10.662452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420aae7-e0ed-49e2-832b-06be47bc6b4f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:10.704028 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccc5f2-cf7f-4a53-b699-524f05ee21ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:10.704877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccc5f2-cf7f-4a53-b699-524f05ee21ee None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:10.705153 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccc5f2-cf7f-4a53-b699-524f05ee21ee None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:10.705479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccc5f2-cf7f-4a53-b699-524f05ee21ee None admin] PATH_INFO: `/v3/services/network` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:10.706149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccc5f2-cf7f-4a53-b699-524f05ee21ee None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:10.707553 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ccc5f2-cf7f-4a53-b699-524f05ee21ee None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:10.709647 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:10.721780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-30ccc5f2-cf7f-4a53-b699-524f05ee21ee None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 25 15:54:10.721780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:10.721780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:10.721780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:10.721780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:10.721780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:10.721780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:10.721780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:10.721780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 25 15:54:10.721780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:10.721780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:10.721780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:10.721780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:10.721780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:10.721780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:10.721780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:10.721780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:10.722824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:10.722824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:10.722824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:10.722824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:10.722824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:10.722824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:10.722824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:10.722824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:10.722824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:10.722824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:10.722824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:10.722824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:10.722824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:10.722824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:10.722824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:10.722824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:10.723665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:10.723665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:10.723665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:10.723665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:10.723665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:10.723665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:10.723665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:10.723665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:10.723665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:10.723665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:10.723665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:10.723665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:10.723665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 25 15:54:10.723665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:10.723665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 379/776] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:10 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:10.729505 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59913eb3-fc96-4130-8bd5-ebf7095de810 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:10.853702 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59913eb3-fc96-4130-8bd5-ebf7095de810 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:10.897810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd37a1-6488-41a0-9c0d-7fb01e3962a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:10.898485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd37a1-6488-41a0-9c0d-7fb01e3962a0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:10.899077 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd37a1-6488-41a0-9c0d-7fb01e3962a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:10.899365 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd37a1-6488-41a0-9c0d-7fb01e3962a0 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:10.900093 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd37a1-6488-41a0-9c0d-7fb01e3962a0 None admin] RBAC: Adding query filter params (name=network) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:10.900420 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd37a1-6488-41a0-9c0d-7fb01e3962a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:10.901820 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd37a1-6488-41a0-9c0d-7fb01e3962a0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:10.907009 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 398/777] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Jun 25 15:54:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:10.912159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccc5f2-cf7f-4a53-b699-524f05ee21ee None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:10.983175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c276648-9de7-4060-b5b3-c239f5982fe1 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:10.983499 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c276648-9de7-4060-b5b3-c239f5982fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:10.983742 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c276648-9de7-4060-b5b3-c239f5982fe1 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:10.985448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 399/778] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:54:10.992060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f4808-7a92-4345-96ba-aba5a37041de None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:10.992838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f4808-7a92-4345-96ba-aba5a37041de None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:10.993331 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f4808-7a92-4345-96ba-aba5a37041de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:11.018592 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-bf4f4808-7a92-4345-96ba-aba5a37041de None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:11.020314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-bf4f4808-7a92-4345-96ba-aba5a37041de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:11.054169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccc5f2-cf7f-4a53-b699-524f05ee21ee None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:11.100512 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:11.117416 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfb054-eaf2-4373-87e9-3b0a52282880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:11.118241 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfb054-eaf2-4373-87e9-3b0a52282880 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:11.118522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfb054-eaf2-4373-87e9-3b0a52282880 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:11.118789 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfb054-eaf2-4373-87e9-3b0a52282880 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:11.119493 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfb054-eaf2-4373-87e9-3b0a52282880 None admin] RBAC: Adding query filter params (type=network) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:11.119792 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfb054-eaf2-4373-87e9-3b0a52282880 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:11.121237 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfb054-eaf2-4373-87e9-3b0a52282880 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:11.128012 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 380/779] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Jun 25 15:54:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:11.129340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0296463-cd08-47f2-940d-0531b76b5480 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:11.129620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0296463-cd08-47f2-940d-0531b76b5480 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:11.129895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0296463-cd08-47f2-940d-0531b76b5480 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:11.131348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 381/780] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:11.134323 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0296463-cd08-47f2-940d-0531b76b5480 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:11.145096 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 400/781] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:11.146518 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fce42b-6592-4aa3-bd9a-66740df850d4 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:11.146808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fce42b-6592-4aa3-bd9a-66740df850d4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:11.147083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fce42b-6592-4aa3-bd9a-66740df850d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:11.165149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-82fce42b-6592-4aa3-bd9a-66740df850d4 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:11.167183 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-82fce42b-6592-4aa3-bd9a-66740df850d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:11.223297 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:11.254122 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 401/782] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:11.255815 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fce42b-6592-4aa3-bd9a-66740df850d4 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:11.283359 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0296463-cd08-47f2-940d-0531b76b5480 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:11.319963 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3560ef-633a-4f87-bb28-e801e3f57912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:11.321046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3560ef-633a-4f87-bb28-e801e3f57912 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:11.321562 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3560ef-633a-4f87-bb28-e801e3f57912 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:11.321562 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3560ef-633a-4f87-bb28-e801e3f57912 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:11.322445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3560ef-633a-4f87-bb28-e801e3f57912 None admin] RBAC: Adding query filter params (interface=public, service_id=84f9119152104673a5fff68666fd8b37, region_id=RegionOne) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:11.322756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3560ef-633a-4f87-bb28-e801e3f57912 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:11.324149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3560ef-633a-4f87-bb28-e801e3f57912 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:11.332450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 382/783] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Jun 25 15:54:11 2022] GET /identity/v3/endpoints?service_id=84f9119152104673a5fff68666fd8b37&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:11.333152 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3560ef-633a-4f87-bb28-e801e3f57912 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:11.394477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fce42b-6592-4aa3-bd9a-66740df850d4 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:11.405432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:11.438369 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7bbc7-18b9-4278-97a0-244879cb438f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:11.439172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7bbc7-18b9-4278-97a0-244879cb438f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:11.439452 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7bbc7-18b9-4278-97a0-244879cb438f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:11.439725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7bbc7-18b9-4278-97a0-244879cb438f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:11.451354 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec7bbc7-18b9-4278-97a0-244879cb438f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:54:11.459100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-6ec7bbc7-18b9-4278-97a0-244879cb438f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:54:11.459100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:11.459100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:11.459100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:11.459100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:11.459100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:11.459100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:11.459100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:11.459100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:11.459100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:11.459100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:11.459100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:11.459100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:54:11.459100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:54:11.459100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:54:11.459100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:54:11.460024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:54:11.460024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:54:11.460024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:54:11.460024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:54:11.460024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:11.460024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:11.460024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:11.460024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:11.460024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:11.460024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:11.460024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:11.460024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:11.460024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:11.460024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:11.460024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:11.460024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:11.460935 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:11.460935 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:11.460935 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:11.460935 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:11.460935 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:54:11.460935 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:54:11.460935 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:54:11.460935 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:54:11.460935 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:54:11.460935 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:54:11.460935 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:54:11.460935 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:11.464686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 402/784] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:54:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:11.465551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7bbc7-18b9-4278-97a0-244879cb438f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:11.473447 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3560ef-633a-4f87-bb28-e801e3f57912 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:11.508829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:11.521183 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b29f29-ae35-4535-aade-fe191cbc6a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:11.521995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b29f29-ae35-4535-aade-fe191cbc6a95 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:11.522275 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b29f29-ae35-4535-aade-fe191cbc6a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:11.523068 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b29f29-ae35-4535-aade-fe191cbc6a95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:11.537786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b29f29-ae35-4535-aade-fe191cbc6a95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:54:11.548053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-a2b29f29-ae35-4535-aade-fe191cbc6a95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:54:11.548053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:11.548053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:11.548053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:11.548053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:11.548053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:11.548053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:11.548053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:11.548053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:11.548053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:11.548053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:11.548053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:11.548053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:54:11.548053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:54:11.548053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:54:11.548053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:54:11.550210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:54:11.550210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:54:11.550210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:54:11.550210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:54:11.550210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:11.550210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:11.550210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:11.550210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:11.550210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:11.550210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:11.550210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:11.550210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:11.550210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:11.550210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:11.550210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:11.550210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:11.552277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:11.552277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:11.552277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:11.552277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:11.552277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:54:11.552277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:54:11.552277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:54:11.552277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:54:11.552277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:54:11.552277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:54:11.552277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:54:11.552277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:11.553780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 383/785] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:54:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:11.553780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b29f29-ae35-4535-aade-fe191cbc6a95 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:11.584850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7bbc7-18b9-4278-97a0-244879cb438f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:11.631165 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e7888-c11c-451d-bae4-ee39c3f2fc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:11.631862 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e7888-c11c-451d-bae4-ee39c3f2fc14 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:11.632141 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e7888-c11c-451d-bae4-ee39c3f2fc14 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:11.632440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e7888-c11c-451d-bae4-ee39c3f2fc14 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:11.633726 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e7888-c11c-451d-bae4-ee39c3f2fc14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:11.635642 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e7888-c11c-451d-bae4-ee39c3f2fc14 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:11.642989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 403/786] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Jun 25 15:54:11 2022] GET /identity/v3/services => generated 772 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:11.643776 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e7888-c11c-451d-bae4-ee39c3f2fc14 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:11.682528 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b29f29-ae35-4535-aade-fe191cbc6a95 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:11.715462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:11.729454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153d491-1d00-438a-abb0-5ce9ba41b816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:11.730196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153d491-1d00-438a-abb0-5ce9ba41b816 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:11.730481 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153d491-1d00-438a-abb0-5ce9ba41b816 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:11.730759 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153d491-1d00-438a-abb0-5ce9ba41b816 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:11.731545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153d491-1d00-438a-abb0-5ce9ba41b816 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:11.731851 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153d491-1d00-438a-abb0-5ce9ba41b816 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:11.733332 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153d491-1d00-438a-abb0-5ce9ba41b816 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:11.743069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 384/787] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:54:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:11.749706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153d491-1d00-438a-abb0-5ce9ba41b816 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:11.859622 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:11.861913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e7888-c11c-451d-bae4-ee39c3f2fc14 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:11.899158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff050e3-755c-4e07-b9a8-23db07bc8001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:11.899846 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff050e3-755c-4e07-b9a8-23db07bc8001 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:11.900129 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff050e3-755c-4e07-b9a8-23db07bc8001 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:11.900434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff050e3-755c-4e07-b9a8-23db07bc8001 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:11.901927 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff050e3-755c-4e07-b9a8-23db07bc8001 None admin] RBAC: Adding query filter params (name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:11.902241 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff050e3-755c-4e07-b9a8-23db07bc8001 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:11.903619 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff050e3-755c-4e07-b9a8-23db07bc8001 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:11.912484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 404/788] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:54:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:11.918424 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff050e3-755c-4e07-b9a8-23db07bc8001 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:11.920880 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153d491-1d00-438a-abb0-5ce9ba41b816 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:11.999742 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed68c07-9378-4a0d-b733-9d9f1cd79dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:12.000895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed68c07-9378-4a0d-b733-9d9f1cd79dcf None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:12.001355 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed68c07-9378-4a0d-b733-9d9f1cd79dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:12.001830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed68c07-9378-4a0d-b733-9d9f1cd79dcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:12.025141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed68c07-9378-4a0d-b733-9d9f1cd79dcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:12.028921 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed68c07-9378-4a0d-b733-9d9f1cd79dcf None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:12.061747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-7ed68c07-9378-4a0d-b733-9d9f1cd79dcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:12.061747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:12.061747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:12.061747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:12.061747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:12.061747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:12.061747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:12.061747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:12.061747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:12.061747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:12.061747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:12.061747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:12.061747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:12.061747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:12.061747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:12.062701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:12.062701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:12.062701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:12.062701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:12.062701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:12.062701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:12.062701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:12.062701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:12.062701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:12.062701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:12.062701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:12.062701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:12.062701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:12.062701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:12.062701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:12.062701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:12.062701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:12.063853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:12.063853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:12.063853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:12.063853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:12.063853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 385/789] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:12.069706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed68c07-9378-4a0d-b733-9d9f1cd79dcf None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:12.080748 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:12.090457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff050e3-755c-4e07-b9a8-23db07bc8001 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:12.156725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a2b19-0b99-4e4e-a144-09e94df2c347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:12.157440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a2b19-0b99-4e4e-a144-09e94df2c347 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:12.157774 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a2b19-0b99-4e4e-a144-09e94df2c347 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:12.158055 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a2b19-0b99-4e4e-a144-09e94df2c347 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:12.176131 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a2b19-0b99-4e4e-a144-09e94df2c347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:12.178726 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a2b19-0b99-4e4e-a144-09e94df2c347 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:12.217058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-9a2a2b19-0b99-4e4e-a144-09e94df2c347 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:12.217058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:12.217058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:12.217058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:12.217058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:12.217058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:12.217058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:12.217058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:12.217058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:12.217058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:12.217058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:12.217058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:12.217058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:12.217058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:12.217058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:12.218142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:12.218142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:12.218142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:12.218142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:12.218142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:12.218142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:12.218142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:12.218142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:12.218142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:12.218142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:12.218142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:12.218142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:12.218142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:12.218142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:12.218142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:12.218142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:12.218142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:12.219274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:12.219274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:12.219274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:12.219274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:12.219274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 405/790] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:12.224031 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a2b19-0b99-4e4e-a144-09e94df2c347 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:12.228709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:12.236146 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed68c07-9378-4a0d-b733-9d9f1cd79dcf None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:12.302023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481cdd9-6ca5-4db1-8311-a8c6f836b80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:12.302948 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481cdd9-6ca5-4db1-8311-a8c6f836b80e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:12.303332 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481cdd9-6ca5-4db1-8311-a8c6f836b80e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:12.303698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481cdd9-6ca5-4db1-8311-a8c6f836b80e None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:12.304548 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481cdd9-6ca5-4db1-8311-a8c6f836b80e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:12.305045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481cdd9-6ca5-4db1-8311-a8c6f836b80e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:12.306626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481cdd9-6ca5-4db1-8311-a8c6f836b80e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:12.331270 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 386/791] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:12.336338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481cdd9-6ca5-4db1-8311-a8c6f836b80e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:12.417227 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a2b19-0b99-4e4e-a144-09e94df2c347 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:12.487984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c9b1a-1fbc-4cdc-a2ba-b53654c8eb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:12.488883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c9b1a-1fbc-4cdc-a2ba-b53654c8eb56 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:12.489094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c9b1a-1fbc-4cdc-a2ba-b53654c8eb56 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:12.489315 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c9b1a-1fbc-4cdc-a2ba-b53654c8eb56 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:12.490051 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c9b1a-1fbc-4cdc-a2ba-b53654c8eb56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:12.490354 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c9b1a-1fbc-4cdc-a2ba-b53654c8eb56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:12.491714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c9b1a-1fbc-4cdc-a2ba-b53654c8eb56 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:12.508745 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481cdd9-6ca5-4db1-8311-a8c6f836b80e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:12.512086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 406/792] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:12.516888 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c9b1a-1fbc-4cdc-a2ba-b53654c8eb56 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:12.532982 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:12.546373 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b575171-dfe0-448e-a000-b8b9dde5f03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:12.547503 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b575171-dfe0-448e-a000-b8b9dde5f03c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:12.547989 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b575171-dfe0-448e-a000-b8b9dde5f03c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:12.548528 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b575171-dfe0-448e-a000-b8b9dde5f03c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:12.556504 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b575171-dfe0-448e-a000-b8b9dde5f03c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:12.558230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b575171-dfe0-448e-a000-b8b9dde5f03c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:12.573377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-9b575171-dfe0-448e-a000-b8b9dde5f03c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 25 15:54:12.573377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:12.573377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:12.573377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:12.573377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:12.573377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:12.573377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:12.573377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:12.573377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:12.573377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:12.573377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:12.573377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:12.573377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:54:12.573377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:54:12.573377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:54:12.573377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:54:12.574365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:12.574365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:12.574365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:54:12.574365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:12.574365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:54:12.574365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:12.574365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:12.574365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:12.574365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:12.574365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:12.574365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:12.574365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:12.574365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:12.574365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:12.574365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:12.574365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:12.574365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:12.574365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:12.575326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:12.575326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:12.575326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:54:12.575326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:54:12.575326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:54:12.575326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:54:12.575326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:54:12.575326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:54:12.575326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 25 15:54:12.575326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:12.579118 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 387/793] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:54:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:12.586717 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b575171-dfe0-448e-a000-b8b9dde5f03c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:12.632858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:12.694048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c9b1a-1fbc-4cdc-a2ba-b53654c8eb56 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:12.761501 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7c33d-270c-4405-afe0-022b31b87097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:12.762387 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7c33d-270c-4405-afe0-022b31b87097 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:12.762662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7c33d-270c-4405-afe0-022b31b87097 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:12.762930 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7c33d-270c-4405-afe0-022b31b87097 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:12.773676 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd7c33d-270c-4405-afe0-022b31b87097 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:12.777185 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd7c33d-270c-4405-afe0-022b31b87097 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:12.782197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b575171-dfe0-448e-a000-b8b9dde5f03c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:12.807600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-8bd7c33d-270c-4405-afe0-022b31b87097 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 25 15:54:12.807600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:12.807600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:12.807600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:12.807600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:12.807600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:12.807600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:12.807600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:12.807600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:12.807600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:12.807600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:12.807600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:12.807600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:54:12.807600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:54:12.807600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:54:12.807600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:54:12.808662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:12.808662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:12.808662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:54:12.808662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:12.808662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:54:12.808662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:12.808662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:12.808662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:12.808662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:12.808662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:12.808662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:12.808662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:12.808662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:12.808662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:12.808662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:12.808662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:12.808662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:12.808662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:12.809630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:12.809630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:12.809630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:54:12.809630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:54:12.809630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:54:12.809630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:54:12.809630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:54:12.809630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:54:12.809630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 25 15:54:12.809630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:12.810809 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 407/794] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Jun 25 15:54:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:12.817766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7c33d-270c-4405-afe0-022b31b87097 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:12.837066 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa1c25-3d12-4f47-9010-5b5bdac7d090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:12.837748 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa1c25-3d12-4f47-9010-5b5bdac7d090 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:12.838024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa1c25-3d12-4f47-9010-5b5bdac7d090 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:12.838290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa1c25-3d12-4f47-9010-5b5bdac7d090 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:12.839131 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa1c25-3d12-4f47-9010-5b5bdac7d090 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:12.839434 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa1c25-3d12-4f47-9010-5b5bdac7d090 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:12.840919 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa1c25-3d12-4f47-9010-5b5bdac7d090 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:12.880515 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 388/795] 10.222.0.13 () {56 vars in 1135 bytes} [Sat Jun 25 15:54:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:12.885876 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa1c25-3d12-4f47-9010-5b5bdac7d090 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:12.907118 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:12.945906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7c33d-270c-4405-afe0-022b31b87097 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:12.987540 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db3bca-5a01-488b-a481-a5602045c435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:12.988669 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db3bca-5a01-488b-a481-a5602045c435 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:12.989854 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db3bca-5a01-488b-a481-a5602045c435 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:12.990304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db3bca-5a01-488b-a481-a5602045c435 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:12.991300 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db3bca-5a01-488b-a481-a5602045c435 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:12.991756 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db3bca-5a01-488b-a481-a5602045c435 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:12.993474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db3bca-5a01-488b-a481-a5602045c435 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:13.015310 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 408/796] 10.222.0.13 () {56 vars in 1149 bytes} [Sat Jun 25 15:54:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:13.017509 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac792d0-e945-4bf4-812f-bdccc8e7db94 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:13.018008 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac792d0-e945-4bf4-812f-bdccc8e7db94 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:13.018490 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac792d0-e945-4bf4-812f-bdccc8e7db94 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:13.019950 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:13.020718 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 409/797] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:13.021602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac792d0-e945-4bf4-812f-bdccc8e7db94 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:13.031507 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa1c25-3d12-4f47-9010-5b5bdac7d090 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:13.078126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8d855-5e72-4c2b-b83a-34ba79c27600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:13.079592 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8d855-5e72-4c2b-b83a-34ba79c27600 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:13.079882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8d855-5e72-4c2b-b83a-34ba79c27600 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:13.080400 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8d855-5e72-4c2b-b83a-34ba79c27600 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:13.094327 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8d855-5e72-4c2b-b83a-34ba79c27600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:13.096398 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8d855-5e72-4c2b-b83a-34ba79c27600 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:13.120162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-d0a8d855-5e72-4c2b-b83a-34ba79c27600 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:13.120162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:13.120162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:13.120162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:13.120162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:13.120162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:13.120162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:13.120162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:13.120162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:13.120162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:13.120162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:13.120162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:13.120162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:13.120162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:13.120162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:13.121262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:13.121262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:13.121262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:13.121262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:13.121262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:13.121262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:13.121262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:13.121262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:13.121262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:13.121262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:13.121262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:13.121262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:13.121262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:13.121262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:13.121262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:13.121262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:13.121262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:13.122162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:13.122162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:13.122162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:13.122162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:13.122162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 389/798] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:13.128462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb31967-5e5b-4b5d-b745-d0d770aea5fe None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:13.128462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb31967-5e5b-4b5d-b745-d0d770aea5fe None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:13.128462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb31967-5e5b-4b5d-b745-d0d770aea5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:13.133341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac792d0-e945-4bf4-812f-bdccc8e7db94 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:13.146580 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2fb31967-5e5b-4b5d-b745-d0d770aea5fe None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:13.147781 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2fb31967-5e5b-4b5d-b745-d0d770aea5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:13.158470 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:13.171834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a3a9f-294e-4804-b0ca-9d8ea8b3d4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:13.172654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a3a9f-294e-4804-b0ca-9d8ea8b3d4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:13.172932 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a3a9f-294e-4804-b0ca-9d8ea8b3d4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:13.173201 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a3a9f-294e-4804-b0ca-9d8ea8b3d4c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:13.183551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a3a9f-294e-4804-b0ca-9d8ea8b3d4c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:13.185032 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a3a9f-294e-4804-b0ca-9d8ea8b3d4c7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:13.202775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-0f3a3a9f-294e-4804-b0ca-9d8ea8b3d4c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:13.202775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:13.202775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:13.202775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:13.202775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:13.202775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:13.202775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:13.202775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:13.202775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:13.202775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:13.202775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:13.202775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:13.202775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:13.202775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:13.202775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:13.203651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:13.203651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:13.203651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:13.203651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:13.203651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:13.203651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:13.203651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:13.203651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:13.203651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:13.203651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:13.203651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:13.203651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:13.203651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:13.203651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:13.203651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:13.203651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:13.203651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:13.204600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:13.204600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:13.204600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:13.204600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:13.204600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 410/799] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:13.205170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c9e3e-0ea6-4508-9278-c4e0872594ce None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:13.205375 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c9e3e-0ea6-4508-9278-c4e0872594ce None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:13.205650 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c9e3e-0ea6-4508-9278-c4e0872594ce None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:13.208542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 411/800] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:13.208542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c9e3e-0ea6-4508-9278-c4e0872594ce None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:13.225723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 390/801] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:13.226879 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb31967-5e5b-4b5d-b745-d0d770aea5fe None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:13.296338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:13.327304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c9e3e-0ea6-4508-9278-c4e0872594ce None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:13.339188 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb31967-5e5b-4b5d-b745-d0d770aea5fe None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:13.363285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f1c5d-16cc-45b3-bffc-6f43f43b3a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:13.363949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f1c5d-16cc-45b3-bffc-6f43f43b3a6b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:13.364224 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f1c5d-16cc-45b3-bffc-6f43f43b3a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:13.364529 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f1c5d-16cc-45b3-bffc-6f43f43b3a6b None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:13.365309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f1c5d-16cc-45b3-bffc-6f43f43b3a6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:13.365604 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f1c5d-16cc-45b3-bffc-6f43f43b3a6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:13.367012 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f1c5d-16cc-45b3-bffc-6f43f43b3a6b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:13.373452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc569a4-76a4-4811-a0b2-6128410f1d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:13.374122 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc569a4-76a4-4811-a0b2-6128410f1d77 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:13.374403 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc569a4-76a4-4811-a0b2-6128410f1d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:13.374675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc569a4-76a4-4811-a0b2-6128410f1d77 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:13.375411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc569a4-76a4-4811-a0b2-6128410f1d77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:13.375723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc569a4-76a4-4811-a0b2-6128410f1d77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:13.377164 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc569a4-76a4-4811-a0b2-6128410f1d77 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:13.379286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 412/802] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:13.381078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1eaca-686a-4dde-a1b0-98b520da36b8 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:13.381538 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1eaca-686a-4dde-a1b0-98b520da36b8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:13.381947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1eaca-686a-4dde-a1b0-98b520da36b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:13.390605 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 391/803] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:13.391868 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc569a4-76a4-4811-a0b2-6128410f1d77 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:13.401121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9ad1eaca-686a-4dde-a1b0-98b520da36b8 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:13.402472 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9ad1eaca-686a-4dde-a1b0-98b520da36b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:13.433662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:13.475863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 413/804] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:13.477734 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1eaca-686a-4dde-a1b0-98b520da36b8 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:13.502743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc569a4-76a4-4811-a0b2-6128410f1d77 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:13.525654 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:13.541188 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bab229-888f-41ec-95fc-85a575bb34a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:13.541990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bab229-888f-41ec-95fc-85a575bb34a9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:13.542303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bab229-888f-41ec-95fc-85a575bb34a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:13.542576 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bab229-888f-41ec-95fc-85a575bb34a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:13.552973 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bab229-888f-41ec-95fc-85a575bb34a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:54:13.560932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-39bab229-888f-41ec-95fc-85a575bb34a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:54:13.560932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:13.560932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:13.560932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:13.560932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:13.560932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:13.560932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:13.560932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:13.560932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:13.560932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:13.560932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:13.560932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:13.560932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:54:13.560932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:54:13.560932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:54:13.560932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:54:13.561861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:54:13.561861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:54:13.561861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:54:13.561861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:54:13.561861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:13.561861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:13.561861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:13.561861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:13.561861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:13.561861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:13.561861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:13.561861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:13.561861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:13.561861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:13.561861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:13.561861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:13.562704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:13.562704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:13.562704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:13.562704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:13.562704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:54:13.562704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:54:13.562704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:54:13.562704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:54:13.562704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:54:13.562704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:54:13.562704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:54:13.562704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:13.567049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 392/805] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:54:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:13.567663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bab229-888f-41ec-95fc-85a575bb34a9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:13.590709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1eaca-686a-4dde-a1b0-98b520da36b8 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:13.625285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225631b-ded9-4068-9bbd-bb599927722c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:13.626090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225631b-ded9-4068-9bbd-bb599927722c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:13.626368 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225631b-ded9-4068-9bbd-bb599927722c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:13.626647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225631b-ded9-4068-9bbd-bb599927722c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:13.633780 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225631b-ded9-4068-9bbd-bb599927722c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:13.635228 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225631b-ded9-4068-9bbd-bb599927722c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:13.649462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-5225631b-ded9-4068-9bbd-bb599927722c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:13.649462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:13.649462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:13.649462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:13.649462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:13.649462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:13.649462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:13.649462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:13.649462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:13.649462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:13.649462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:13.649462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:13.649462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:54:13.649462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:54:13.649462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:54:13.650382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:54:13.650382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:13.650382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:13.650382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:13.650382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:13.650382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:13.650382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:13.650382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:13.650382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:13.650382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:13.650382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:13.650382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:13.650382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:13.650382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:13.650382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:13.650382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:13.650382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:13.651314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:54:13.651314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:54:13.651314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:54:13.651314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:54:13.651314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:54:13.651314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:54:13.651314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:13.651314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:13.651314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 414/806] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:13.652684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225631b-ded9-4068-9bbd-bb599927722c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:13.679073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:13.679961 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bab229-888f-41ec-95fc-85a575bb34a9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:13.715153 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3f5c6-05e1-4640-a805-f3c0e8e7e275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:13.715949 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3f5c6-05e1-4640-a805-f3c0e8e7e275 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:13.716226 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3f5c6-05e1-4640-a805-f3c0e8e7e275 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:13.716532 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3f5c6-05e1-4640-a805-f3c0e8e7e275 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:13.723395 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3f5c6-05e1-4640-a805-f3c0e8e7e275 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:13.724880 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3f5c6-05e1-4640-a805-f3c0e8e7e275 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:13.739154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-f0d3f5c6-05e1-4640-a805-f3c0e8e7e275 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:13.739154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:13.739154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:13.739154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:13.739154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:13.739154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:13.739154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:13.739154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:13.739154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:13.739154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:13.739154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:13.739154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:13.739154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:54:13.739154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:54:13.739154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:54:13.740073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:54:13.740073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:13.740073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:13.740073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:13.740073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:13.740073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:13.740073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:13.740073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:13.740073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:13.740073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:13.740073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:13.740073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:13.740073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:13.740073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:13.740073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:13.740073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:13.740073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:13.741011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:54:13.741011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:54:13.741011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:54:13.741011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:54:13.741011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:54:13.741011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:54:13.741011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:13.741011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:13.741011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 393/807] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:13.741497 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3f5c6-05e1-4640-a805-f3c0e8e7e275 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:13.759241 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225631b-ded9-4068-9bbd-bb599927722c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:13.787922 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:13.794275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8254baf7-af23-4f3b-aa37-204aa6ee4a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:13.795312 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254baf7-af23-4f3b-aa37-204aa6ee4a07 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:13.795738 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254baf7-af23-4f3b-aa37-204aa6ee4a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:13.796175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254baf7-af23-4f3b-aa37-204aa6ee4a07 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:13.797018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8254baf7-af23-4f3b-aa37-204aa6ee4a07 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:13.798534 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8254baf7-af23-4f3b-aa37-204aa6ee4a07 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:13.811089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-8254baf7-af23-4f3b-aa37-204aa6ee4a07 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 25 15:54:13.811089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:13.811089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:13.811089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:13.811089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:13.811089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:13.811089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:13.811089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:13.811089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 25 15:54:13.811089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:13.811089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:13.811089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:13.811089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:13.811089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:13.811089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:13.811089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:13.812329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:13.812329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:13.812329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:13.812329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:13.812329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:13.812329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:13.812329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:13.812329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:13.812329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:13.812329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:13.812329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:13.812329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:13.812329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:13.812329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:13.812329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:13.812329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:13.813389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:13.813389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:13.813389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:13.813389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:13.813389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:13.813389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:13.813389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:13.813389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:13.813389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:13.813389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:13.813389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:13.813389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:13.813389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:13.813389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 25 15:54:13.813389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:13.815612 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 415/808] 10.222.0.13 () {56 vars in 1129 bytes} [Sat Jun 25 15:54:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:13.816585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8254baf7-af23-4f3b-aa37-204aa6ee4a07 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:13.852785 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3f5c6-05e1-4640-a805-f3c0e8e7e275 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:13.891130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f98153-cd81-4057-ac30-007ded54d2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:13.892104 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f98153-cd81-4057-ac30-007ded54d2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:13.892379 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f98153-cd81-4057-ac30-007ded54d2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:13.892699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f98153-cd81-4057-ac30-007ded54d2b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:13.893436 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f98153-cd81-4057-ac30-007ded54d2b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:13.893737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f98153-cd81-4057-ac30-007ded54d2b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:13.895106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f98153-cd81-4057-ac30-007ded54d2b0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:13.903447 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 394/809] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:54:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:13.904189 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f98153-cd81-4057-ac30-007ded54d2b0 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:13.910974 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:13.920946 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8254baf7-af23-4f3b-aa37-204aa6ee4a07 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:13.957004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d97c3f-d524-4ab8-ab24-d621b8092c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:13.957654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d97c3f-d524-4ab8-ab24-d621b8092c91 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:13.957930 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d97c3f-d524-4ab8-ab24-d621b8092c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:13.958211 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d97c3f-d524-4ab8-ab24-d621b8092c91 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:13.958988 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d97c3f-d524-4ab8-ab24-d621b8092c91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:13.959290 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d97c3f-d524-4ab8-ab24-d621b8092c91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:13.960733 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d97c3f-d524-4ab8-ab24-d621b8092c91 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:13.972461 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 416/810] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:54:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:13.973233 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d97c3f-d524-4ab8-ab24-d621b8092c91 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:14.002510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f98153-cd81-4057-ac30-007ded54d2b0 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:14.004115 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:14.037397 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c7053f-8bce-42c6-9e7e-412e9ae0c482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:14.038060 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c7053f-8bce-42c6-9e7e-412e9ae0c482 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:14.038350 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c7053f-8bce-42c6-9e7e-412e9ae0c482 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:14.038612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c7053f-8bce-42c6-9e7e-412e9ae0c482 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:14.039579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c7053f-8bce-42c6-9e7e-412e9ae0c482 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:14.040078 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c7053f-8bce-42c6-9e7e-412e9ae0c482 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:14.041574 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c7053f-8bce-42c6-9e7e-412e9ae0c482 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:14.053341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 395/811] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:54:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:14.058484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4bb8a-7934-4d4b-b012-235a12d7be41 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:14.058484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4bb8a-7934-4d4b-b012-235a12d7be41 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:14.058484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4bb8a-7934-4d4b-b012-235a12d7be41 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:14.058484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 396/812] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:14.058484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4bb8a-7934-4d4b-b012-235a12d7be41 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:14.079303 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d97c3f-d524-4ab8-ab24-d621b8092c91 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:14.115014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878476f-fd07-4d17-9399-65bc162a495b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:14.115795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878476f-fd07-4d17-9399-65bc162a495b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:14.116075 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878476f-fd07-4d17-9399-65bc162a495b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:14.116348 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878476f-fd07-4d17-9399-65bc162a495b None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:14.117081 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878476f-fd07-4d17-9399-65bc162a495b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:14.117468 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878476f-fd07-4d17-9399-65bc162a495b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:14.118838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878476f-fd07-4d17-9399-65bc162a495b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:14.123444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 417/813] 10.222.0.13 () {56 vars in 1124 bytes} [Sat Jun 25 15:54:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:14.123982 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878476f-fd07-4d17-9399-65bc162a495b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:14.138984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:14.161315 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4bb8a-7934-4d4b-b012-235a12d7be41 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:14.194924 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d98ba4-3d27-48b6-bef4-4956528e2ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:14.195833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d98ba4-3d27-48b6-bef4-4956528e2ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:14.196221 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d98ba4-3d27-48b6-bef4-4956528e2ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:14.196531 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d98ba4-3d27-48b6-bef4-4956528e2ef7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:14.206751 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d98ba4-3d27-48b6-bef4-4956528e2ef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:14.208211 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d98ba4-3d27-48b6-bef4-4956528e2ef7 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:14.223812 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878476f-fd07-4d17-9399-65bc162a495b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:14.226295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-b7d98ba4-3d27-48b6-bef4-4956528e2ef7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:14.226295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:14.226295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:14.226295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:14.226295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:14.226295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:14.226295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:14.226295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:14.226295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:14.226295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:14.226295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:14.226295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:14.226295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:14.226295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:14.226295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:14.227342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:14.227342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:14.227342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:14.227342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:14.227342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:14.227342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:14.227342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:14.227342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:14.227342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:14.227342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:14.227342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:14.227342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:14.227342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:14.227342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:14.227342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:14.227342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:14.227342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:14.228244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:14.228244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:14.228244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:14.228244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:14.228244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 397/814] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:14.228911 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d9d61-341e-4adb-be89-c6206deee180 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:14.229199 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d9d61-341e-4adb-be89-c6206deee180 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:14.229475 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d9d61-341e-4adb-be89-c6206deee180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:14.246342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-106d9d61-341e-4adb-be89-c6206deee180 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:14.247564 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-106d9d61-341e-4adb-be89-c6206deee180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:14.256930 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:14.257043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51bf50-3b9d-428b-a379-347c67a98a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:14.257971 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51bf50-3b9d-428b-a379-347c67a98a6c None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:14.258258 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51bf50-3b9d-428b-a379-347c67a98a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:14.258665 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51bf50-3b9d-428b-a379-347c67a98a6c None admin] PATH_INFO: `/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/19788c4239ca44f9a31a7d32086c8947/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:14.260824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51bf50-3b9d-428b-a379-347c67a98a6c None admin] RBAC: Authorizing `identity:create_grant(project_id=3600648eb94b434b8c797d2f454e46b8, user_id=19788c4239ca44f9a31a7d32086c8947, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:14.262253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51bf50-3b9d-428b-a379-347c67a98a6c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:14.274929 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 418/815] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:54:14 2022] PUT /identity/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/19788c4239ca44f9a31a7d32086c8947/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:54:14.275680 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51bf50-3b9d-428b-a379-347c67a98a6c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:14.328696 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 398/816] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:14.328696 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d9d61-341e-4adb-be89-c6206deee180 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:14.377067 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:14.405173 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51bf50-3b9d-428b-a379-347c67a98a6c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:14.448134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8d4d1-c199-4604-8bce-ef73778f1d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:14.449194 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8d4d1-c199-4604-8bce-ef73778f1d17 None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:14.449419 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8d4d1-c199-4604-8bce-ef73778f1d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:14.449615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d9d61-341e-4adb-be89-c6206deee180 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:14.449716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8d4d1-c199-4604-8bce-ef73778f1d17 None admin] PATH_INFO: `/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/28b3e73439e64050b406a9f3e9b0160a/roles/7f1996bcf7e54638b4f53b3d8ba83495` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:14.465373 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8d4d1-c199-4604-8bce-ef73778f1d17 None admin] RBAC: Authorizing `identity:create_grant(project_id=3600648eb94b434b8c797d2f454e46b8, user_id=28b3e73439e64050b406a9f3e9b0160a, role_id=7f1996bcf7e54638b4f53b3d8ba83495)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:14.466816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e8d4d1-c199-4604-8bce-ef73778f1d17 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:14.484653 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 419/817] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:54:14 2022] PUT /identity/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/28b3e73439e64050b406a9f3e9b0160a/roles/7f1996bcf7e54638b4f53b3d8ba83495 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:54:14.484653 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8d4d1-c199-4604-8bce-ef73778f1d17 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:14.494320 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a8f4a-d182-491c-9e01-0efd87c07d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:14.495322 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a8f4a-d182-491c-9e01-0efd87c07d5a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:14.495738 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a8f4a-d182-491c-9e01-0efd87c07d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:14.496851 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a8f4a-d182-491c-9e01-0efd87c07d5a None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:14.497626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a8f4a-d182-491c-9e01-0efd87c07d5a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:14.497934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a8f4a-d182-491c-9e01-0efd87c07d5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:14.499739 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a8f4a-d182-491c-9e01-0efd87c07d5a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:14.505941 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 399/818] 10.222.0.13 () {56 vars in 1124 bytes} [Sat Jun 25 15:54:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:14.507146 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a8f4a-d182-491c-9e01-0efd87c07d5a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:14.520191 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:14.628522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a8f4a-d182-491c-9e01-0efd87c07d5a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:14.635297 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8d4d1-c199-4604-8bce-ef73778f1d17 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:14.668007 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:14.673164 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daffa8a9-9287-4675-a205-ca1ebb3db71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:14.673476 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffa8a9-9287-4675-a205-ca1ebb3db71d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:14.673766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffa8a9-9287-4675-a205-ca1ebb3db71d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:14.674040 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffa8a9-9287-4675-a205-ca1ebb3db71d None admin] PATH_INFO: `/v3/services/network` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:14.674699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daffa8a9-9287-4675-a205-ca1ebb3db71d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:14.676107 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daffa8a9-9287-4675-a205-ca1ebb3db71d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:14.681352 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fecbc31-74be-4d1c-b983-108ddfba0816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:14.682040 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecbc31-74be-4d1c-b983-108ddfba0816 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:14.682714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecbc31-74be-4d1c-b983-108ddfba0816 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:14.683330 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecbc31-74be-4d1c-b983-108ddfba0816 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:14.685375 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fecbc31-74be-4d1c-b983-108ddfba0816 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:14.685375 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fecbc31-74be-4d1c-b983-108ddfba0816 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:14.686163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fecbc31-74be-4d1c-b983-108ddfba0816 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:14.690129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-daffa8a9-9287-4675-a205-ca1ebb3db71d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 25 15:54:14.690129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:14.690129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:14.690129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:14.690129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:14.690129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:14.690129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:14.690129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:14.690129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 25 15:54:14.690129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:14.690129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:14.690129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:14.690129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:14.690129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:14.690129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:14.690129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:14.690129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:14.691454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:14.691454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:14.691454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:14.691454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:14.691454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:14.691454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:14.691454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:14.691454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:14.691454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:14.691454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:14.691454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:14.691454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:14.691454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:14.691454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:14.691454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:14.691454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:14.692369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:14.692369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:14.692369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:14.692369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:14.692369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:14.692369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:14.692369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:14.692369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:14.692369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:14.692369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:14.692369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:14.692369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:14.692369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 25 15:54:14.692369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:14.692369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 400/819] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:14 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:14.693281 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daffa8a9-9287-4675-a205-ca1ebb3db71d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:14.704394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 420/820] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:14.705651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fecbc31-74be-4d1c-b983-108ddfba0816 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:14.768901 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:14.808536 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daffa8a9-9287-4675-a205-ca1ebb3db71d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:14.816053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fecbc31-74be-4d1c-b983-108ddfba0816 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:14.849657 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bd7fb-9aab-4540-bc55-465aa4066a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:14.850455 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bd7fb-9aab-4540-bc55-465aa4066a2f None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:14.850738 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bd7fb-9aab-4540-bc55-465aa4066a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:14.851009 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bd7fb-9aab-4540-bc55-465aa4066a2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:14.851917 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448bd7fb-9aab-4540-bc55-465aa4066a2f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:14.853418 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448bd7fb-9aab-4540-bc55-465aa4066a2f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:14.867861 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33f64c-5d63-4524-9e96-8dbf622a06fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:14.868929 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33f64c-5d63-4524-9e96-8dbf622a06fa None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:14.869478 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33f64c-5d63-4524-9e96-8dbf622a06fa None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:14.870044 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef33f64c-5d63-4524-9e96-8dbf622a06fa None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:14.871136 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33f64c-5d63-4524-9e96-8dbf622a06fa None admin] RBAC: Adding query filter params (name=network) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:14.871686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33f64c-5d63-4524-9e96-8dbf622a06fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:14.873351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef33f64c-5d63-4524-9e96-8dbf622a06fa None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:14.879257 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 421/822] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Jun 25 15:54:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:14.880462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33f64c-5d63-4524-9e96-8dbf622a06fa None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:14.882994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 401/822] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Jun 25 15:54:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:54:14.884779 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bd7fb-9aab-4540-bc55-465aa4066a2f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:14.935919 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:15.009668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef33f64c-5d63-4524-9e96-8dbf622a06fa None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:15.079250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1318123-f2fc-4b8f-bf50-3113a998b4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:15.080439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1318123-f2fc-4b8f-bf50-3113a998b4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:15.080948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1318123-f2fc-4b8f-bf50-3113a998b4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:15.082047 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1318123-f2fc-4b8f-bf50-3113a998b4d3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:15.092228 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1318123-f2fc-4b8f-bf50-3113a998b4d3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:15.094413 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1318123-f2fc-4b8f-bf50-3113a998b4d3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:15.101588 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bd7fb-9aab-4540-bc55-465aa4066a2f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:15.112073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-b1318123-f2fc-4b8f-bf50-3113a998b4d3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 25 15:54:15.112073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:15.112073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:15.112073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:15.112073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:15.112073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:15.112073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:15.112073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:15.112073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:15.112073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:15.112073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:15.112073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:15.112073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:54:15.112073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:54:15.112073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:54:15.112073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:54:15.113110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:15.113110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:15.113110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:54:15.113110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:15.113110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:54:15.113110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:15.113110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:15.113110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:15.113110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:15.113110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:15.113110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:15.113110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:15.113110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:15.113110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:15.113110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:15.113110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:15.113110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:15.113110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:15.114050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:15.114050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:15.114050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:54:15.114050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:54:15.114050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:54:15.114050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:54:15.114050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:54:15.114050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:54:15.114050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 25 15:54:15.114050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:15.116856 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 422/823] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Jun 25 15:54:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:15.122624 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1318123-f2fc-4b8f-bf50-3113a998b4d3 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:15.133503 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:15.142614 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187b08a-d7ab-4bd1-a946-c35358ee9b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:15.143345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187b08a-d7ab-4bd1-a946-c35358ee9b86 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:15.143626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187b08a-d7ab-4bd1-a946-c35358ee9b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:15.144074 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187b08a-d7ab-4bd1-a946-c35358ee9b86 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:15.145151 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187b08a-d7ab-4bd1-a946-c35358ee9b86 None admin] RBAC: Adding query filter params (type=network) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:15.145769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187b08a-d7ab-4bd1-a946-c35358ee9b86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:15.147507 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187b08a-d7ab-4bd1-a946-c35358ee9b86 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:15.153175 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 402/824] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Jun 25 15:54:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:15.161182 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187b08a-d7ab-4bd1-a946-c35358ee9b86 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:15.240698 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1318123-f2fc-4b8f-bf50-3113a998b4d3 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:15.248978 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:15.283296 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b89f22-96cc-47f3-bf17-518d623663f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:15.283874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187b08a-d7ab-4bd1-a946-c35358ee9b86 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:15.284011 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b89f22-96cc-47f3-bf17-518d623663f5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:15.284246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b89f22-96cc-47f3-bf17-518d623663f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:15.284580 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b89f22-96cc-47f3-bf17-518d623663f5 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:15.285431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b89f22-96cc-47f3-bf17-518d623663f5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:15.285746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b89f22-96cc-47f3-bf17-518d623663f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:15.287176 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b89f22-96cc-47f3-bf17-518d623663f5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:15.306364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 423/825] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Jun 25 15:54:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:15.311639 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b89f22-96cc-47f3-bf17-518d623663f5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:15.330250 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82e469-df80-4194-9ae0-7727cc3a892e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:15.331443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82e469-df80-4194-9ae0-7727cc3a892e None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:15.331921 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82e469-df80-4194-9ae0-7727cc3a892e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:15.332343 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82e469-df80-4194-9ae0-7727cc3a892e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:15.333271 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82e469-df80-4194-9ae0-7727cc3a892e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:15.335221 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82e469-df80-4194-9ae0-7727cc3a892e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:15.368451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 403/826] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Jun 25 15:54:15 2022] POST /identity/v3/endpoints => generated 327 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:54:15.477331 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b89f22-96cc-47f3-bf17-518d623663f5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:15.537511 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f67846-e31c-4851-b0e5-4e5d0871f59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:15.538171 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f67846-e31c-4851-b0e5-4e5d0871f59b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:15.538447 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f67846-e31c-4851-b0e5-4e5d0871f59b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:15.538715 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f67846-e31c-4851-b0e5-4e5d0871f59b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:15.575408 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:15.627582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f67846-e31c-4851-b0e5-4e5d0871f59b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:15.629059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f67846-e31c-4851-b0e5-4e5d0871f59b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:15.734373 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 424/827] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:54:15 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:54:15.739889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82e469-df80-4194-9ae0-7727cc3a892e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:15.813081 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:15.885695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82e469-df80-4194-9ae0-7727cc3a892e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:15.921939 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33962e0b-6477-4647-aad9-212958a5cce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:15.922743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33962e0b-6477-4647-aad9-212958a5cce7 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:15.923019 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33962e0b-6477-4647-aad9-212958a5cce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:15.923290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33962e0b-6477-4647-aad9-212958a5cce7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:15.934400 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33962e0b-6477-4647-aad9-212958a5cce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:15.935868 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33962e0b-6477-4647-aad9-212958a5cce7 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:15.953149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-33962e0b-6477-4647-aad9-212958a5cce7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:15.953149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:15.953149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:15.953149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:15.953149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:15.953149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:15.953149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:15.953149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:15.953149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:15.953149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:15.953149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:15.953149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:15.953149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:15.953149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:15.953149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:15.954086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:15.954086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:15.954086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:15.954086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:15.954086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:15.954086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:15.954086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:15.954086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:15.954086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:15.954086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:15.954086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:15.954086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:15.954086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:15.954086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:15.954086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:15.954086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:15.954086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:15.954973 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:15.954973 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:15.954973 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:15.954973 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:15.954973 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 404/828] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:15.961491 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f67846-e31c-4851-b0e5-4e5d0871f59b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:16.021563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:16.066354 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f67846-e31c-4851-b0e5-4e5d0871f59b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:16.101994 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adaf3e-f7f6-49db-b92a-e2b09fdd53b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:16.102667 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adaf3e-f7f6-49db-b92a-e2b09fdd53b0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:16.102953 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adaf3e-f7f6-49db-b92a-e2b09fdd53b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:16.103223 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4adaf3e-f7f6-49db-b92a-e2b09fdd53b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:16.103954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adaf3e-f7f6-49db-b92a-e2b09fdd53b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:16.104265 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adaf3e-f7f6-49db-b92a-e2b09fdd53b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:16.105670 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4adaf3e-f7f6-49db-b92a-e2b09fdd53b0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:16.117859 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 425/829] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:16.123289 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33962e0b-6477-4647-aad9-212958a5cce7 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:16.220727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:16.223603 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33962e0b-6477-4647-aad9-212958a5cce7 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:16.257694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a8605-7121-420f-9b0c-2a281af7415f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:16.258557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a8605-7121-420f-9b0c-2a281af7415f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:16.259047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a8605-7121-420f-9b0c-2a281af7415f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:16.259355 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a8605-7121-420f-9b0c-2a281af7415f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:16.266948 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a8605-7121-420f-9b0c-2a281af7415f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:16.268428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a8605-7121-420f-9b0c-2a281af7415f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:16.283401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-3a8a8605-7121-420f-9b0c-2a281af7415f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:16.283401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:16.283401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:16.283401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:16.283401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:16.283401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:16.283401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:16.283401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:16.283401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:16.283401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:16.283401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:16.283401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:16.283401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:54:16.283401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:54:16.283401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:54:16.284292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:54:16.284292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:16.284292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:16.284292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:16.284292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:16.284292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:16.284292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:16.284292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:16.284292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:16.284292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:16.284292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:16.284292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:16.284292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:16.284292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:16.284292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:16.284292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:16.284292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:16.285317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:54:16.285317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:54:16.285317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:54:16.285317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:54:16.285317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:54:16.285317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:54:16.285317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:16.285317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:16.285317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 405/830] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:16.290586 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adaf3e-f7f6-49db-b92a-e2b09fdd53b0 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:16.394016 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4adaf3e-f7f6-49db-b92a-e2b09fdd53b0 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:16.411620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:16.429470 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4a666-fa0a-4ec1-a1b5-383897ada794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:16.430210 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4a666-fa0a-4ec1-a1b5-383897ada794 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:16.430648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4a666-fa0a-4ec1-a1b5-383897ada794 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:16.430931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4a666-fa0a-4ec1-a1b5-383897ada794 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:16.431725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4a666-fa0a-4ec1-a1b5-383897ada794 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:16.432029 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4a666-fa0a-4ec1-a1b5-383897ada794 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:16.433476 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4a666-fa0a-4ec1-a1b5-383897ada794 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:16.446127 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 426/831] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:54:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:16.452252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a8605-7121-420f-9b0c-2a281af7415f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:16.489291 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf859c-0d47-462b-9f08-49d2b6d52d95 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:16.489483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf859c-0d47-462b-9f08-49d2b6d52d95 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:16.489759 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf859c-0d47-462b-9f08-49d2b6d52d95 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:16.491402 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 427/832] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:16.497536 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc7a70-35de-4248-a583-93e55454051d None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:16.497832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc7a70-35de-4248-a583-93e55454051d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:16.498112 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc7a70-35de-4248-a583-93e55454051d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:16.514671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-accc7a70-35de-4248-a583-93e55454051d None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:16.515912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-accc7a70-35de-4248-a583-93e55454051d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:16.568932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a8605-7121-420f-9b0c-2a281af7415f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:16.592394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 428/833] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:16.607257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d167172-5b9c-463e-ac29-3c5b20c31b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:16.607916 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d167172-5b9c-463e-ac29-3c5b20c31b1d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:16.608197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d167172-5b9c-463e-ac29-3c5b20c31b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:16.608496 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d167172-5b9c-463e-ac29-3c5b20c31b1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:16.609330 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d167172-5b9c-463e-ac29-3c5b20c31b1d None admin] RBAC: Adding query filter params (role.id=b1e13c449fa64b3e88b6832bb767214b, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=3622c5ccb7534e7c8e8342b4f6643bc3) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:16.609638 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d167172-5b9c-463e-ac29-3c5b20c31b1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:16.611098 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d167172-5b9c-463e-ac29-3c5b20c31b1d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:16.616492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc7a70-35de-4248-a583-93e55454051d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:16.617685 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 406/834] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:54:16 2022] GET /identity/v3/role_assignments?user.id=3622c5ccb7534e7c8e8342b4f6643bc3&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:16.727737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fff3b7-8720-4306-a8c9-b823b0019732 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:16.728072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fff3b7-8720-4306-a8c9-b823b0019732 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:16.728259 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fff3b7-8720-4306-a8c9-b823b0019732 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:16.729929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 407/835] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:54:16.735397 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907276d-075d-45a8-a43a-ace2104f13ce None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:16.735679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907276d-075d-45a8-a43a-ace2104f13ce None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:16.735953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907276d-075d-45a8-a43a-ace2104f13ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:16.738851 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc7a70-35de-4248-a583-93e55454051d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:16.750567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:16.756252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d907276d-075d-45a8-a43a-ace2104f13ce None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:16.757699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d907276d-075d-45a8-a43a-ace2104f13ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:16.779682 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:16.781501 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c8e0e-76ea-4f3e-914e-5e23613619a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:16.782318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c8e0e-76ea-4f3e-914e-5e23613619a8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:16.782661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c8e0e-76ea-4f3e-914e-5e23613619a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:16.782939 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c8e0e-76ea-4f3e-914e-5e23613619a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:16.796845 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c8e0e-76ea-4f3e-914e-5e23613619a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:54:16.804489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-294c8e0e-76ea-4f3e-914e-5e23613619a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:54:16.804489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:16.804489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:16.804489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:16.804489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:16.804489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:16.804489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:16.804489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:16.804489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:16.804489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:16.804489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:16.804489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:16.804489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:54:16.804489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:54:16.804489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:54:16.804489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:54:16.805997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:54:16.805997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:54:16.805997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:54:16.805997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:54:16.805997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:16.805997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:16.805997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:16.805997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:16.805997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:16.805997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:16.805997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:16.805997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:16.805997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:16.805997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:16.805997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:16.805997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:16.807207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:16.807207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:16.807207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:16.807207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:16.807207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:54:16.807207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:54:16.807207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:54:16.807207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:54:16.807207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:54:16.807207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:54:16.807207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:54:16.807207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:16.807207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 429/836] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:54:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:16.817208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c8e0e-76ea-4f3e-914e-5e23613619a8 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:16.850855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 408/837] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:16.880055 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907276d-075d-45a8-a43a-ace2104f13ce None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:16.956197 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c8e0e-76ea-4f3e-914e-5e23613619a8 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:16.992115 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c97ce-9694-4557-bf6a-b753eb0f45ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:16.993867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c97ce-9694-4557-bf6a-b753eb0f45ba None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:16.994342 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c97ce-9694-4557-bf6a-b753eb0f45ba None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:16.994869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c97ce-9694-4557-bf6a-b753eb0f45ba None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:16.995808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c97ce-9694-4557-bf6a-b753eb0f45ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:16.996304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c97ce-9694-4557-bf6a-b753eb0f45ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:16.997416 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907276d-075d-45a8-a43a-ace2104f13ce None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:16.998106 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c97ce-9694-4557-bf6a-b753eb0f45ba None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:17.008319 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 430/838] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:54:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:17.015248 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c97ce-9694-4557-bf6a-b753eb0f45ba None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:17.040351 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87977501-ee5a-4c3b-881e-c7383ce01a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:17.041229 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87977501-ee5a-4c3b-881e-c7383ce01a5e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:17.041507 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87977501-ee5a-4c3b-881e-c7383ce01a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:17.041773 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87977501-ee5a-4c3b-881e-c7383ce01a5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:17.052818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87977501-ee5a-4c3b-881e-c7383ce01a5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:54:17.057995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:17.061498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-87977501-ee5a-4c3b-881e-c7383ce01a5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:54:17.061498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:17.061498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:17.061498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:17.061498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:17.061498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:17.061498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:17.061498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:17.061498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:17.061498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:17.061498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:17.061498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:17.061498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:54:17.061498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:54:17.061498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:54:17.061498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:54:17.062451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:54:17.062451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:54:17.062451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:54:17.062451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:54:17.062451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:17.062451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:17.062451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:17.062451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:17.062451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:17.062451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:17.062451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:17.062451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:17.062451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:17.062451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:17.062451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:17.062451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:17.063290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:17.063290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:17.063290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:17.063290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:17.063290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:54:17.063290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:54:17.063290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:54:17.063290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:54:17.063290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:54:17.063290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:54:17.063290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:54:17.063290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:17.063290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 409/839] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:54:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:17.069101 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87977501-ee5a-4c3b-881e-c7383ce01a5e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:17.086381 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:17.157952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c97ce-9694-4557-bf6a-b753eb0f45ba None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:17.193202 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87977501-ee5a-4c3b-881e-c7383ce01a5e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:17.204024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ebec9-595c-4f71-bd28-a7f07ee41407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:17.204854 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ebec9-595c-4f71-bd28-a7f07ee41407 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:17.205133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ebec9-595c-4f71-bd28-a7f07ee41407 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:17.205408 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ebec9-595c-4f71-bd28-a7f07ee41407 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:17.224023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ebec9-595c-4f71-bd28-a7f07ee41407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:17.227020 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ebec9-595c-4f71-bd28-a7f07ee41407 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:17.236344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776b339-eb7d-4216-bbe4-b366bf977e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:17.237356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776b339-eb7d-4216-bbe4-b366bf977e3a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:17.237835 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776b339-eb7d-4216-bbe4-b366bf977e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:17.238305 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776b339-eb7d-4216-bbe4-b366bf977e3a None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:17.239726 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776b339-eb7d-4216-bbe4-b366bf977e3a None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:17.240254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776b339-eb7d-4216-bbe4-b366bf977e3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:17.242456 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776b339-eb7d-4216-bbe4-b366bf977e3a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:17.256285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 410/840] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:54:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:17.256494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-5b0ebec9-595c-4f71-bd28-a7f07ee41407 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:17.256494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:17.256494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:17.256494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:17.256494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:17.256494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:17.256494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:17.256494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:17.256494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:17.256494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:17.256494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:17.256494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:17.256494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:17.256494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:17.256494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:17.257366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:17.257366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:17.257366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:17.257366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:17.257366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:17.257366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:17.257366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:17.257366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:17.257366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:17.257366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:17.257366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:17.257366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:17.257366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:17.257366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:17.257366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:17.257366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:17.257366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:17.258388 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:17.258388 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:17.258388 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:17.258388 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:17.260261 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776b339-eb7d-4216-bbe4-b366bf977e3a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:17.263483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 431/841] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:17.267138 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ebec9-595c-4f71-bd28-a7f07ee41407 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:17.354091 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:17.358592 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:17.380048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ebec9-595c-4f71-bd28-a7f07ee41407 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:17.381929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776b339-eb7d-4216-bbe4-b366bf977e3a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:17.417641 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680000f-fa71-46b6-a68a-2742eeb7280d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:17.418686 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c653781-dc4b-41e9-b8da-9e8d07be32c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:17.419009 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680000f-fa71-46b6-a68a-2742eeb7280d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:17.419485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c653781-dc4b-41e9-b8da-9e8d07be32c9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:17.419592 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680000f-fa71-46b6-a68a-2742eeb7280d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:17.419766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c653781-dc4b-41e9-b8da-9e8d07be32c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:17.420040 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c653781-dc4b-41e9-b8da-9e8d07be32c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:17.420494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680000f-fa71-46b6-a68a-2742eeb7280d None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:17.421570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680000f-fa71-46b6-a68a-2742eeb7280d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:17.422161 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680000f-fa71-46b6-a68a-2742eeb7280d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:17.423811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680000f-fa71-46b6-a68a-2742eeb7280d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:17.431752 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c653781-dc4b-41e9-b8da-9e8d07be32c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:17.433237 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c653781-dc4b-41e9-b8da-9e8d07be32c9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:17.438302 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 432/842] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:17.439955 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e0a0a-e7fc-495c-bbec-5f3f2b31e63a None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:17.440434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e0a0a-e7fc-495c-bbec-5f3f2b31e63a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:17.440875 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e0a0a-e7fc-495c-bbec-5f3f2b31e63a None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:17.442617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 433/843] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:17.444697 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e0a0a-e7fc-495c-bbec-5f3f2b31e63a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:17.453017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-7c653781-dc4b-41e9-b8da-9e8d07be32c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:17.453017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:17.453017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:17.453017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:17.453017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:17.453017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:17.453017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:17.453017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:17.453017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:17.453017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:17.453017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:17.453017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:17.453017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:17.453017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:17.453017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:17.453957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:17.453957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:17.453957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:17.453957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:17.453957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:17.453957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:17.453957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:17.453957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:17.453957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:17.453957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:17.453957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:17.453957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:17.453957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:17.453957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:17.453957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:17.453957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:17.453957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:17.459709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:17.459709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:17.459709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:17.459709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:17.459709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 411/844] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:17.459709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a70f8-46c3-4008-9a26-bfff7b9b0036 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:17.459709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a70f8-46c3-4008-9a26-bfff7b9b0036 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:17.459709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a70f8-46c3-4008-9a26-bfff7b9b0036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:17.478236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f12a70f8-46c3-4008-9a26-bfff7b9b0036 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:17.479680 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f12a70f8-46c3-4008-9a26-bfff7b9b0036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:17.559963 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e0a0a-e7fc-495c-bbec-5f3f2b31e63a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:17.600224 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5527c4-3af5-4f9e-b2f0-6ca6385f6fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:17.601072 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5527c4-3af5-4f9e-b2f0-6ca6385f6fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:17.601355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5527c4-3af5-4f9e-b2f0-6ca6385f6fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:17.601699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5527c4-3af5-4f9e-b2f0-6ca6385f6fd2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:17.610470 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5527c4-3af5-4f9e-b2f0-6ca6385f6fd2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:17.611979 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5527c4-3af5-4f9e-b2f0-6ca6385f6fd2 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:17.628246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-2e5527c4-3af5-4f9e-b2f0-6ca6385f6fd2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 25 15:54:17.628246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:17.628246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:17.628246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:17.628246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:17.628246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:17.628246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:17.628246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:17.628246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:17.628246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:17.628246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:17.628246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:17.628246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:54:17.628246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:54:17.628246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:54:17.628246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:54:17.629484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:17.629484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:17.629484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:54:17.629484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:17.629484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:54:17.629484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:17.629484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:17.629484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:17.629484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:17.629484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:17.629484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:17.629484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:17.629484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:17.629484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:17.629484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:17.629484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:17.629484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:17.629484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:17.630424 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:17.630424 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:17.630424 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:54:17.630424 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:54:17.630424 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:54:17.630424 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:54:17.630424 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:54:17.630424 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:54:17.630424 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 25 15:54:17.630424 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:17.632773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 434/845] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:54:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:17.634329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5527c4-3af5-4f9e-b2f0-6ca6385f6fd2 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:17.637649 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:17.676566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 412/846] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:17.677935 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a70f8-46c3-4008-9a26-bfff7b9b0036 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:17.738946 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5527c4-3af5-4f9e-b2f0-6ca6385f6fd2 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:17.742755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:17.778383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514d890-55c3-4d85-8d71-d3f78730018a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:17.779073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514d890-55c3-4d85-8d71-d3f78730018a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:17.779357 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514d890-55c3-4d85-8d71-d3f78730018a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:17.779629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514d890-55c3-4d85-8d71-d3f78730018a None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:17.780357 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514d890-55c3-4d85-8d71-d3f78730018a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:17.780779 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514d890-55c3-4d85-8d71-d3f78730018a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:17.782144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514d890-55c3-4d85-8d71-d3f78730018a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:17.787369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a70f8-46c3-4008-9a26-bfff7b9b0036 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:17.794263 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 435/847] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:17.795073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514d890-55c3-4d85-8d71-d3f78730018a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:17.827042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606de99-d714-43ae-839a-521c2ac3ae46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:17.827668 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606de99-d714-43ae-839a-521c2ac3ae46 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:17.827950 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606de99-d714-43ae-839a-521c2ac3ae46 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:17.828278 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606de99-d714-43ae-839a-521c2ac3ae46 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:17.829149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606de99-d714-43ae-839a-521c2ac3ae46 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:17.829452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606de99-d714-43ae-839a-521c2ac3ae46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:17.830893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606de99-d714-43ae-839a-521c2ac3ae46 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:17.855146 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 413/848] 10.222.0.13 () {56 vars in 1135 bytes} [Sat Jun 25 15:54:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:17.856158 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606de99-d714-43ae-839a-521c2ac3ae46 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:17.869485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:17.910135 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514d890-55c3-4d85-8d71-d3f78730018a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:17.944816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e441717-fc28-4d48-a2c2-febf4dadfd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:17.945813 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e441717-fc28-4d48-a2c2-febf4dadfd92 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:17.946264 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e441717-fc28-4d48-a2c2-febf4dadfd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:17.946549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e441717-fc28-4d48-a2c2-febf4dadfd92 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:17.947206 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e441717-fc28-4d48-a2c2-febf4dadfd92 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:17.948610 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e441717-fc28-4d48-a2c2-febf4dadfd92 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:17.960238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-5e441717-fc28-4d48-a2c2-febf4dadfd92 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 25 15:54:17.960238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:17.960238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:17.960238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:17.960238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:17.960238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:17.960238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:17.960238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:17.960238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 25 15:54:17.960238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:17.960238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:17.960238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:17.960238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:17.960238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:17.960238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:17.960238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:17.960238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:17.961313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:17.961313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:17.961313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:17.961313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:17.961313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:17.961313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:17.961313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:17.961313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:17.961313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:17.961313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:17.961313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:17.961313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:17.961313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:17.961313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:17.961313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:17.961313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:17.962149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:17.962149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:17.962149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:17.962149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:17.962149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:17.962149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:17.962149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:17.962149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:17.962149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:17.962149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:17.962149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:17.962149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:17.962149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 25 15:54:17.962149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:17.962149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 436/849] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:54:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:17.962952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e441717-fc28-4d48-a2c2-febf4dadfd92 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:17.972381 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606de99-d714-43ae-839a-521c2ac3ae46 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:17.990183 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:18.009729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009cfff-50d4-4d63-bd2b-06165a17d029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:18.010545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009cfff-50d4-4d63-bd2b-06165a17d029 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:18.010826 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009cfff-50d4-4d63-bd2b-06165a17d029 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:18.011101 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009cfff-50d4-4d63-bd2b-06165a17d029 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:18.019633 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009cfff-50d4-4d63-bd2b-06165a17d029 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:18.021129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009cfff-50d4-4d63-bd2b-06165a17d029 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:18.036159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-0009cfff-50d4-4d63-bd2b-06165a17d029 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 25 15:54:18.036159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:18.036159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:18.036159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:18.036159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:18.036159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:18.036159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:18.036159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:18.036159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:18.036159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:18.036159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:18.036159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:18.036159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:54:18.036159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:54:18.036159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:54:18.036159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:54:18.037303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:18.037303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:18.037303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:54:18.037303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:18.037303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:54:18.037303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:18.037303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:18.037303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:18.037303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:18.037303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:18.037303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:18.037303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:18.037303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:18.037303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:18.037303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:18.037303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:18.037303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:18.037303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:18.038324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:18.038324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:18.038324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:54:18.038324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:54:18.038324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:54:18.038324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:54:18.038324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:54:18.038324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:54:18.038324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 25 15:54:18.038324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:18.042244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 414/850] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Jun 25 15:54:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:18.043352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009cfff-50d4-4d63-bd2b-06165a17d029 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:18.065126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e441717-fc28-4d48-a2c2-febf4dadfd92 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:18.091024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:18.100196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae1fb3-3c38-422a-b3f4-d1f6278f2d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:18.100961 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae1fb3-3c38-422a-b3f4-d1f6278f2d7f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:18.101318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae1fb3-3c38-422a-b3f4-d1f6278f2d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:18.101710 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ae1fb3-3c38-422a-b3f4-d1f6278f2d7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:18.145366 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009cfff-50d4-4d63-bd2b-06165a17d029 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:18.170065 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae1fb3-3c38-422a-b3f4-d1f6278f2d7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:18.171646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ae1fb3-3c38-422a-b3f4-d1f6278f2d7f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:18.179091 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4aeb7b-ae70-4bfd-901c-92d86340bce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:18.179969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4aeb7b-ae70-4bfd-901c-92d86340bce3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:18.180322 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4aeb7b-ae70-4bfd-901c-92d86340bce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:18.180712 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4aeb7b-ae70-4bfd-901c-92d86340bce3 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:18.181511 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4aeb7b-ae70-4bfd-901c-92d86340bce3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:18.181885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4aeb7b-ae70-4bfd-901c-92d86340bce3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:18.183363 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4aeb7b-ae70-4bfd-901c-92d86340bce3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:18.193449 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 415/851] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:54:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:18.193449 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4aeb7b-ae70-4bfd-901c-92d86340bce3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:18.241060 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:18.306081 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4aeb7b-ae70-4bfd-901c-92d86340bce3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:18.342275 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c598fa8-a4fc-47ee-842d-6e1c5415afdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:18.342873 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c598fa8-a4fc-47ee-842d-6e1c5415afdd None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:18.343153 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c598fa8-a4fc-47ee-842d-6e1c5415afdd None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:18.343425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c598fa8-a4fc-47ee-842d-6e1c5415afdd None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:18.344732 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c598fa8-a4fc-47ee-842d-6e1c5415afdd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:18.345055 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c598fa8-a4fc-47ee-842d-6e1c5415afdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:18.346763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c598fa8-a4fc-47ee-842d-6e1c5415afdd None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:18.366151 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 416/852] 10.222.0.13 () {56 vars in 1149 bytes} [Sat Jun 25 15:54:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:18.367036 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c598fa8-a4fc-47ee-842d-6e1c5415afdd None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:18.398198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 437/853] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:54:17 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:54:18.399489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae1fb3-3c38-422a-b3f4-d1f6278f2d7f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:18.427479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:18.482918 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:18.487094 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c598fa8-a4fc-47ee-842d-6e1c5415afdd None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:18.501836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ae1fb3-3c38-422a-b3f4-d1f6278f2d7f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:18.524522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab056a1-7773-454d-9b36-b5807521554f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:18.525199 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab056a1-7773-454d-9b36-b5807521554f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:18.525479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab056a1-7773-454d-9b36-b5807521554f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:18.525756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab056a1-7773-454d-9b36-b5807521554f None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:18.526464 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab056a1-7773-454d-9b36-b5807521554f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:18.526882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab056a1-7773-454d-9b36-b5807521554f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:18.528150 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab056a1-7773-454d-9b36-b5807521554f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:18.540461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 417/854] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:54:18 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:18.540461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab056a1-7773-454d-9b36-b5807521554f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:18.546820 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac0af3-169b-4138-bfd2-80c6544a9c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:18.547583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac0af3-169b-4138-bfd2-80c6544a9c17 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:18.547934 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac0af3-169b-4138-bfd2-80c6544a9c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:18.548280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac0af3-169b-4138-bfd2-80c6544a9c17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:18.631729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac0af3-169b-4138-bfd2-80c6544a9c17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:18.633607 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac0af3-169b-4138-bfd2-80c6544a9c17 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:18.669188 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:18.683574 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab056a1-7773-454d-9b36-b5807521554f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:18.725742 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03a706-0a5c-44b5-ad78-440098fcea6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:18.726589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03a706-0a5c-44b5-ad78-440098fcea6e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:18.727519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03a706-0a5c-44b5-ad78-440098fcea6e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:18.727519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03a706-0a5c-44b5-ad78-440098fcea6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:18.740903 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03a706-0a5c-44b5-ad78-440098fcea6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:18.742485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03a706-0a5c-44b5-ad78-440098fcea6e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:18.763389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-cd03a706-0a5c-44b5-ad78-440098fcea6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:18.763389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:18.763389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:18.763389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:18.763389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:18.763389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:18.763389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:18.763389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:18.763389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:18.763389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:18.763389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:18.763389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:18.763389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:18.763389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:18.763389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:18.764338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:18.764338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:18.764338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:18.764338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:18.764338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:18.764338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:18.764338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:18.764338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:18.764338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:18.764338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:18.764338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:18.764338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:18.764338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:18.764338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:18.764338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:18.764338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:18.764338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:18.765276 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:18.765276 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:18.765276 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:18.765276 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:18.765276 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 418/855] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:18.770748 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03a706-0a5c-44b5-ad78-440098fcea6e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:18.777048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 438/856] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:54:18 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:54:18.782743 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac0af3-169b-4138-bfd2-80c6544a9c17 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:18.788307 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:18.881114 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03a706-0a5c-44b5-ad78-440098fcea6e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:18.889052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac0af3-169b-4138-bfd2-80c6544a9c17 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:18.916089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:18.919796 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83f209-0389-464e-9c11-89d3da16b70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:18.920492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83f209-0389-464e-9c11-89d3da16b70d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:18.920831 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83f209-0389-464e-9c11-89d3da16b70d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:18.921144 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83f209-0389-464e-9c11-89d3da16b70d None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:18.921882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83f209-0389-464e-9c11-89d3da16b70d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:18.922187 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83f209-0389-464e-9c11-89d3da16b70d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:18.923561 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83f209-0389-464e-9c11-89d3da16b70d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:18.925641 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf8be6-021b-4394-92b5-c78a394f5183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:18.926428 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf8be6-021b-4394-92b5-c78a394f5183 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:18.926714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf8be6-021b-4394-92b5-c78a394f5183 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:18.926980 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf8be6-021b-4394-92b5-c78a394f5183 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:18.936190 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 419/857] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:18.937447 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf8be6-021b-4394-92b5-c78a394f5183 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:18.938883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf8be6-021b-4394-92b5-c78a394f5183 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:18.941694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b669ffc-ec46-49e7-a79a-c5f52d25f443 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:18.941984 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b669ffc-ec46-49e7-a79a-c5f52d25f443 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:18.942261 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b669ffc-ec46-49e7-a79a-c5f52d25f443 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:18.948277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 420/858] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:18.948277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b669ffc-ec46-49e7-a79a-c5f52d25f443 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:18.959523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-a2cf8be6-021b-4394-92b5-c78a394f5183 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:18.959523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:18.959523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:18.959523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:18.959523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:18.959523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:18.959523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:18.959523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:18.959523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:18.959523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:18.959523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:18.959523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:18.959523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:18.959523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:18.959523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:18.960487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:18.960487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:18.960487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:18.960487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:18.960487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:18.960487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:18.960487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:18.960487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:18.960487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:18.960487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:18.960487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:18.960487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:18.960487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:18.960487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:18.960487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:18.960487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:18.960487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:18.962128 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:18.962128 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:18.962128 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:18.962128 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:18.965489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 439/859] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:18.966792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83970c8-62d7-426e-8e9f-c37912f73227 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:18.967079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83970c8-62d7-426e-8e9f-c37912f73227 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:18.967362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83970c8-62d7-426e-8e9f-c37912f73227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:18.984934 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e83970c8-62d7-426e-8e9f-c37912f73227 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:18.986271 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e83970c8-62d7-426e-8e9f-c37912f73227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:19.032241 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:19.068468 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 440/860] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:19.069037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b669ffc-ec46-49e7-a79a-c5f52d25f443 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:19.069137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83970c8-62d7-426e-8e9f-c37912f73227 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:19.101856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88ed96-b5d7-4ad9-bd9e-f6ca679a63e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:19.102821 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88ed96-b5d7-4ad9-bd9e-f6ca679a63e6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:19.103097 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88ed96-b5d7-4ad9-bd9e-f6ca679a63e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:19.103361 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88ed96-b5d7-4ad9-bd9e-f6ca679a63e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:19.110247 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88ed96-b5d7-4ad9-bd9e-f6ca679a63e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:19.111691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88ed96-b5d7-4ad9-bd9e-f6ca679a63e6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:19.126312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-ae88ed96-b5d7-4ad9-bd9e-f6ca679a63e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:19.126312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:19.126312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:19.126312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:19.126312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:19.126312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:19.126312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:19.126312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:19.126312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:19.126312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:19.126312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:19.126312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:19.126312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:54:19.126312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:54:19.126312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:54:19.127262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:54:19.127262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:19.127262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:19.127262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:19.127262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:19.127262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:19.127262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:19.127262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:19.127262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:19.127262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:19.127262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:19.127262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:19.127262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:19.127262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:19.127262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:19.127262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:19.127262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:19.128240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:54:19.128240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:54:19.128240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:54:19.128240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:54:19.128240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:54:19.128240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:54:19.128240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:19.128240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:19.128240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 421/861] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:19.128240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88ed96-b5d7-4ad9-bd9e-f6ca679a63e6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:19.184805 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83970c8-62d7-426e-8e9f-c37912f73227 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:19.196727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:19.223618 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26c418-4200-4666-80cd-d0753937263b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:19.224282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26c418-4200-4666-80cd-d0753937263b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:19.224597 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26c418-4200-4666-80cd-d0753937263b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:19.224879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26c418-4200-4666-80cd-d0753937263b None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:19.225605 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26c418-4200-4666-80cd-d0753937263b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:19.225908 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26c418-4200-4666-80cd-d0753937263b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:19.227332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26c418-4200-4666-80cd-d0753937263b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:19.228180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88ed96-b5d7-4ad9-bd9e-f6ca679a63e6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:19.239421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 441/862] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:19.241579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26c418-4200-4666-80cd-d0753937263b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:19.264588 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0e6b5-8952-4cd2-9890-b002569ff5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:19.265411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0e6b5-8952-4cd2-9890-b002569ff5de None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:19.265688 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0e6b5-8952-4cd2-9890-b002569ff5de None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:19.265961 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0e6b5-8952-4cd2-9890-b002569ff5de None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:19.266947 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0e6b5-8952-4cd2-9890-b002569ff5de None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:19.268322 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0e6b5-8952-4cd2-9890-b002569ff5de None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:19.270141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:19.282428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-49e0e6b5-8952-4cd2-9890-b002569ff5de None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 25 15:54:19.282428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:19.282428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:19.282428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:19.282428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:19.282428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:19.282428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:19.282428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:19.282428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 25 15:54:19.282428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:19.282428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:19.282428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:19.282428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:19.282428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:19.282428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:19.282428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:19.283590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:19.283590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:19.283590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:19.283590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:19.283590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:19.283590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:19.283590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:19.283590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:19.283590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:19.283590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:19.283590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:19.283590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:19.283590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:19.283590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:19.283590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:19.283590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:19.284485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:19.284485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:19.284485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:19.284485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:19.284485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:19.284485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:19.284485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:19.284485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:19.284485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:19.284485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:19.284485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:19.284485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:19.284485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:19.284485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 25 15:54:19.284485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:19.284485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 422/863] 10.222.0.13 () {56 vars in 1132 bytes} [Sat Jun 25 15:54:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:19.285368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0e6b5-8952-4cd2-9890-b002569ff5de None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:19.357158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26c418-4200-4666-80cd-d0753937263b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:19.388741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0e6b5-8952-4cd2-9890-b002569ff5de None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:19.394477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9bed8-efce-4d9c-a485-959ee8d7c7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:19.395296 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9bed8-efce-4d9c-a485-959ee8d7c7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:19.396093 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9bed8-efce-4d9c-a485-959ee8d7c7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:19.396560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9bed8-efce-4d9c-a485-959ee8d7c7e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:19.397807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9bed8-efce-4d9c-a485-959ee8d7c7e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:19.398260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9bed8-efce-4d9c-a485-959ee8d7c7e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:19.400684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9bed8-efce-4d9c-a485-959ee8d7c7e5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:19.413691 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 442/864] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:54:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:19.414534 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9bed8-efce-4d9c-a485-959ee8d7c7e5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:19.431291 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f91da8f-45ab-42d0-848e-32f9550fdb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:19.432212 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f91da8f-45ab-42d0-848e-32f9550fdb24 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:19.432641 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f91da8f-45ab-42d0-848e-32f9550fdb24 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:19.433007 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f91da8f-45ab-42d0-848e-32f9550fdb24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:19.440101 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f91da8f-45ab-42d0-848e-32f9550fdb24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:19.441650 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f91da8f-45ab-42d0-848e-32f9550fdb24 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:19.441860 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:19.456637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-4f91da8f-45ab-42d0-848e-32f9550fdb24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:19.456637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:19.456637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:19.456637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:19.456637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:19.456637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:19.456637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:19.456637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:19.456637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:19.456637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:19.456637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:19.456637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:19.456637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:54:19.456637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:54:19.456637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:54:19.457710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:54:19.457710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:19.457710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:19.457710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:19.457710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:19.457710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:19.457710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:19.457710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:19.457710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:19.457710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:19.457710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:19.457710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:19.457710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:19.457710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:19.457710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:19.457710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:19.457710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:19.458782 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:54:19.458782 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:54:19.458782 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:54:19.458782 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:54:19.458782 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:54:19.458782 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:54:19.458782 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:19.458782 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:19.459775 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 423/865] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:19.465210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f91da8f-45ab-42d0-848e-32f9550fdb24 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:19.521510 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9bed8-efce-4d9c-a485-959ee8d7c7e5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:19.531579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:19.558768 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2f5e5-d658-444f-90a8-d82f51983e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:19.559550 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2f5e5-d658-444f-90a8-d82f51983e98 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:19.559828 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2f5e5-d658-444f-90a8-d82f51983e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:19.560100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2f5e5-d658-444f-90a8-d82f51983e98 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:19.560838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2f5e5-d658-444f-90a8-d82f51983e98 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:19.561143 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2f5e5-d658-444f-90a8-d82f51983e98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:19.562675 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2f5e5-d658-444f-90a8-d82f51983e98 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:19.572188 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 443/866] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Jun 25 15:54:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:19.572188 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2f5e5-d658-444f-90a8-d82f51983e98 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:19.578418 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f91da8f-45ab-42d0-848e-32f9550fdb24 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:19.613095 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb63268-17d4-4c8a-b0c8-83e316864428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:19.613851 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb63268-17d4-4c8a-b0c8-83e316864428 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:19.614132 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb63268-17d4-4c8a-b0c8-83e316864428 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:19.614401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb63268-17d4-4c8a-b0c8-83e316864428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:19.615234 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb63268-17d4-4c8a-b0c8-83e316864428 None admin] RBAC: Adding query filter params (role.id=b1e13c449fa64b3e88b6832bb767214b, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=19788c4239ca44f9a31a7d32086c8947) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:19.615541 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb63268-17d4-4c8a-b0c8-83e316864428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:19.617021 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb63268-17d4-4c8a-b0c8-83e316864428 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:19.624178 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 424/867] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:54:19 2022] GET /identity/v3/role_assignments?user.id=19788c4239ca44f9a31a7d32086c8947&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:19.625026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb63268-17d4-4c8a-b0c8-83e316864428 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:19.677505 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:19.690256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2f5e5-d658-444f-90a8-d82f51983e98 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:19.730338 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb665803-dfc6-40a0-ab4d-4466a45178d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:19.731185 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb665803-dfc6-40a0-ab4d-4466a45178d7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:19.731715 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb665803-dfc6-40a0-ab4d-4466a45178d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:19.732157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb665803-dfc6-40a0-ab4d-4466a45178d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:19.733145 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb665803-dfc6-40a0-ab4d-4466a45178d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:19.733735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb665803-dfc6-40a0-ab4d-4466a45178d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:19.736163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb665803-dfc6-40a0-ab4d-4466a45178d7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:19.752203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 444/868] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:54:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:19.758047 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb665803-dfc6-40a0-ab4d-4466a45178d7 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:19.774540 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb63268-17d4-4c8a-b0c8-83e316864428 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:19.807272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:19.813669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12e8e8-de4a-4f5d-a1f2-d55b679634d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:19.814608 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12e8e8-de4a-4f5d-a1f2-d55b679634d9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:19.814898 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12e8e8-de4a-4f5d-a1f2-d55b679634d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:19.815169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12e8e8-de4a-4f5d-a1f2-d55b679634d9 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:19.815889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12e8e8-de4a-4f5d-a1f2-d55b679634d9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:19.816193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12e8e8-de4a-4f5d-a1f2-d55b679634d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:19.817607 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12e8e8-de4a-4f5d-a1f2-d55b679634d9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:19.823695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 425/869] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Jun 25 15:54:19 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:19.876153 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb665803-dfc6-40a0-ab4d-4466a45178d7 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:19.914833 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e800452-97ed-478d-8711-b61223fbb234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:19.915507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e800452-97ed-478d-8711-b61223fbb234 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:19.915823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e800452-97ed-478d-8711-b61223fbb234 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:19.916052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e800452-97ed-478d-8711-b61223fbb234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:19.916926 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e800452-97ed-478d-8711-b61223fbb234 None admin] RBAC: Adding query filter params (role.id=7f1996bcf7e54638b4f53b3d8ba83495, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=28b3e73439e64050b406a9f3e9b0160a) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:19.917237 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e800452-97ed-478d-8711-b61223fbb234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:19.918687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e800452-97ed-478d-8711-b61223fbb234 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:19.925943 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 445/870] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:54:19 2022] GET /identity/v3/role_assignments?user.id=28b3e73439e64050b406a9f3e9b0160a&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=7f1996bcf7e54638b4f53b3d8ba83495 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:20.604151 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef063a39-d30b-4ff1-8a13-d2c27b916f6d None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:20.604698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef063a39-d30b-4ff1-8a13-d2c27b916f6d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:20.604777 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef063a39-d30b-4ff1-8a13-d2c27b916f6d None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:20.606390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 426/871] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:20.611664 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6e729-b754-471e-b687-88445f161d01 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:20.612079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6e729-b754-471e-b687-88445f161d01 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:20.612237 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6e729-b754-471e-b687-88445f161d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:20.630046 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-17a6e729-b754-471e-b687-88445f161d01 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:20.631229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-17a6e729-b754-471e-b687-88445f161d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:20.643086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:20.703301 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 446/872] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:20.726086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef063a39-d30b-4ff1-8a13-d2c27b916f6d None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:20.835475 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef063a39-d30b-4ff1-8a13-d2c27b916f6d None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:20.870816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0784927-767e-42a1-be0b-97f928985c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:20.871484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0784927-767e-42a1-be0b-97f928985c28 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:20.871759 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0784927-767e-42a1-be0b-97f928985c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:20.872031 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0784927-767e-42a1-be0b-97f928985c28 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:20.872799 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0784927-767e-42a1-be0b-97f928985c28 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:20.874175 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0784927-767e-42a1-be0b-97f928985c28 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:20.890986 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 427/873] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Jun 25 15:54:20 2022] POST /identity/v3/services => generated 251 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:54:21.766675 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9726f3-5aa8-44f1-8e77-758ab82f486e None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:21.767208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9726f3-5aa8-44f1-8e77-758ab82f486e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:21.767208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9726f3-5aa8-44f1-8e77-758ab82f486e None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:21.768827 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 447/874] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:21.774404 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec078af-f748-49ec-bfd4-f6e6bbae3a56 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:21.775263 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec078af-f748-49ec-bfd4-f6e6bbae3a56 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:21.775732 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec078af-f748-49ec-bfd4-f6e6bbae3a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:21.793006 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5ec078af-f748-49ec-bfd4-f6e6bbae3a56 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:21.794492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5ec078af-f748-49ec-bfd4-f6e6bbae3a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:21.842149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:21.870263 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 428/875] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:21 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:21.892730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9726f3-5aa8-44f1-8e77-758ab82f486e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:21.939247 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d28fdf-cb6d-4045-ac37-4c69880494e8 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:21.939873 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d28fdf-cb6d-4045-ac37-4c69880494e8 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:21.940479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d28fdf-cb6d-4045-ac37-4c69880494e8 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:21.942288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 429/876] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:54:21.947530 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee034494-b264-4ead-a83b-34f1f2606390 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:21.948085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee034494-b264-4ead-a83b-34f1f2606390 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:21.948651 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee034494-b264-4ead-a83b-34f1f2606390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:21.965299 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ee034494-b264-4ead-a83b-34f1f2606390 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:21.966822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ee034494-b264-4ead-a83b-34f1f2606390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:22.005471 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9726f3-5aa8-44f1-8e77-758ab82f486e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:22.046876 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae821d86-3140-477f-a448-f76d97468b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:22.047916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae821d86-3140-477f-a448-f76d97468b5a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:22.048353 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae821d86-3140-477f-a448-f76d97468b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:22.048870 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae821d86-3140-477f-a448-f76d97468b5a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:22.049698 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae821d86-3140-477f-a448-f76d97468b5a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:22.051230 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae821d86-3140-477f-a448-f76d97468b5a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:22.052479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 430/877] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:21 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:22.057308 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d013a-6e2b-4f35-8d9d-05ba8435c3a7 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:22.057817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d013a-6e2b-4f35-8d9d-05ba8435c3a7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:22.058266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d013a-6e2b-4f35-8d9d-05ba8435c3a7 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:22.060034 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 431/878] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:22.065446 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c9a7d-a46d-4bce-9352-fa2f3ce80cbb None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:22.065955 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c9a7d-a46d-4bce-9352-fa2f3ce80cbb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:22.066453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c9a7d-a46d-4bce-9352-fa2f3ce80cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:22.067913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-ae821d86-3140-477f-a448-f76d97468b5a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 25 15:54:22.067913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:22.067913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:22.067913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:22.067913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:22.067913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:22.067913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:22.067913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:22.067913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 25 15:54:22.067913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:22.067913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:22.067913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:22.067913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:22.067913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:22.067913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:22.067913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:22.067913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:22.068948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:22.068948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:22.068948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:22.068948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:22.068948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:22.068948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:22.068948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:22.068948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:22.068948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:22.068948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:22.068948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:22.068948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:22.068948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:22.068948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:22.068948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:22.068948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:22.073382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:22.073382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:22.073382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:22.073382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:22.073382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:22.073382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:22.073382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:22.073382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:22.073382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:22.073382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:22.073382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:22.073382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:22.073382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 25 15:54:22.073382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:22.073382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 448/879] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Jun 25 15:54:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:22.075431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae821d86-3140-477f-a448-f76d97468b5a None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:22.088766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:22.090301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e21c9a7d-a46d-4bce-9352-fa2f3ce80cbb None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:22.091728 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e21c9a7d-a46d-4bce-9352-fa2f3ce80cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:22.159051 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 432/880] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:22 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:22.159891 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c9a7d-a46d-4bce-9352-fa2f3ce80cbb None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:22.190177 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae821d86-3140-477f-a448-f76d97468b5a None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:22.190646 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:22.223995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1821318-a348-4cbb-9eb0-52a035df3c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:22.224689 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1821318-a348-4cbb-9eb0-52a035df3c9b None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:22.224972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1821318-a348-4cbb-9eb0-52a035df3c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:22.225235 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1821318-a348-4cbb-9eb0-52a035df3c9b None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:22.225946 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1821318-a348-4cbb-9eb0-52a035df3c9b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:22.227402 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1821318-a348-4cbb-9eb0-52a035df3c9b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:22.242530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 449/881] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Jun 25 15:54:22 2022] POST /identity/v3/services => generated 269 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:54:22.249643 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1821318-a348-4cbb-9eb0-52a035df3c9b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:22.269032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c9a7d-a46d-4bce-9352-fa2f3ce80cbb None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:22.317225 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd24d79-ba5e-437b-8950-a77d8b4f73f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:22.317899 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd24d79-ba5e-437b-8950-a77d8b4f73f0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:22.318074 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd24d79-ba5e-437b-8950-a77d8b4f73f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:22.318342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd24d79-ba5e-437b-8950-a77d8b4f73f0 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:22.319125 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd24d79-ba5e-437b-8950-a77d8b4f73f0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:22.319424 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd24d79-ba5e-437b-8950-a77d8b4f73f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:22.321084 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd24d79-ba5e-437b-8950-a77d8b4f73f0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:22.331440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 433/882] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Jun 25 15:54:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:22.332517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd24d79-ba5e-437b-8950-a77d8b4f73f0 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:22.353516 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:22.410741 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1821318-a348-4cbb-9eb0-52a035df3c9b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:22.451425 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676bfc9-6c76-40b4-9c76-225f9231fbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:22.452626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676bfc9-6c76-40b4-9c76-225f9231fbd7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:22.453128 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676bfc9-6c76-40b4-9c76-225f9231fbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:22.453463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676bfc9-6c76-40b4-9c76-225f9231fbd7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:22.456705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:22.468031 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676bfc9-6c76-40b4-9c76-225f9231fbd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:54:22.469927 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd24d79-ba5e-437b-8950-a77d8b4f73f0 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:22.478314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-3676bfc9-6c76-40b4-9c76-225f9231fbd7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 25 15:54:22.478314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:22.478314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:22.478314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:22.478314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:22.478314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:22.478314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:22.478314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:22.478314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:22.478314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:22.478314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:22.478314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:22.478314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:54:22.478314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:54:22.478314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:54:22.478314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:54:22.479282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:54:22.479282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:54:22.479282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:54:22.479282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:54:22.479282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:22.479282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:22.479282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:22.479282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:22.479282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:22.479282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:22.479282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:22.479282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:22.479282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:22.479282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:22.479282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:22.479282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:22.480120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:22.480120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:22.480120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:22.480120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:22.480120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:54:22.480120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:54:22.480120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:54:22.480120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:54:22.480120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:54:22.480120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:54:22.480120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 25 15:54:22.480120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:22.480120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 450/883] 10.222.0.13 () {56 vars in 1120 bytes} [Sat Jun 25 15:54:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:22.485684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676bfc9-6c76-40b4-9c76-225f9231fbd7 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:22.513240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee97b6-07d0-466c-af6d-9e36f3e567a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:22.514283 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee97b6-07d0-466c-af6d-9e36f3e567a2 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:22.514753 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee97b6-07d0-466c-af6d-9e36f3e567a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:22.515230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee97b6-07d0-466c-af6d-9e36f3e567a2 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:22.516124 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee97b6-07d0-466c-af6d-9e36f3e567a2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:22.516952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee97b6-07d0-466c-af6d-9e36f3e567a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:22.518511 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee97b6-07d0-466c-af6d-9e36f3e567a2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:22.524034 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 434/884] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Jun 25 15:54:22 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:22.598898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676bfc9-6c76-40b4-9c76-225f9231fbd7 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:22.631926 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:22.638898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6ea1b-e851-42f7-87df-f09708b80dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:22.639583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6ea1b-e851-42f7-87df-f09708b80dee None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:22.639873 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6ea1b-e851-42f7-87df-f09708b80dee None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:22.640151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6ea1b-e851-42f7-87df-f09708b80dee None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:22.640945 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6ea1b-e851-42f7-87df-f09708b80dee None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:22.641560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6ea1b-e851-42f7-87df-f09708b80dee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:22.642963 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6ea1b-e851-42f7-87df-f09708b80dee None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:22.652875 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 451/885] 10.222.0.13 () {56 vars in 1115 bytes} [Sat Jun 25 15:54:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:22.658506 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee97b6-07d0-466c-af6d-9e36f3e567a2 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:22.763148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee97b6-07d0-466c-af6d-9e36f3e567a2 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:22.800866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3efba7-767a-489c-b069-2d8c979d9a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:22.802479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3efba7-767a-489c-b069-2d8c979d9a6b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:22.802975 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3efba7-767a-489c-b069-2d8c979d9a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:22.803436 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3efba7-767a-489c-b069-2d8c979d9a6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:22.814530 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3efba7-767a-489c-b069-2d8c979d9a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:22.816479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3efba7-767a-489c-b069-2d8c979d9a6b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:22.834954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-6d3efba7-767a-489c-b069-2d8c979d9a6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:22.834954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:22.834954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:22.834954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:22.834954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:22.834954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:22.834954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:22.834954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:22.834954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:22.834954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:22.834954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:22.834954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:22.834954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:22.834954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:22.834954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:22.836046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:22.836046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:22.836046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:22.836046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:22.836046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:22.836046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:22.836046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:22.836046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:22.836046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:22.836046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:22.836046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:22.836046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:22.836046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:22.836046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:22.836046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:22.836046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:22.836046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:22.837178 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:22.837178 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:22.837178 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:22.837178 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:22.839491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 435/886] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:22.847446 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6ea1b-e851-42f7-87df-f09708b80dee None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:22.941587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58501a-361e-4593-a71b-bd19c5ab513a None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:22.942226 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58501a-361e-4593-a71b-bd19c5ab513a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:22.942696 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58501a-361e-4593-a71b-bd19c5ab513a None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:22.944525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 436/887] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:54:22.949579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f730cae-8cce-4812-a6dd-fd8afdb3e3c7 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:22.950134 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f730cae-8cce-4812-a6dd-fd8afdb3e3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:22.950642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f730cae-8cce-4812-a6dd-fd8afdb3e3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:22.958051 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6ea1b-e851-42f7-87df-f09708b80dee None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:22.971076 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-3f730cae-8cce-4812-a6dd-fd8afdb3e3c7 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:22.973105 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3f730cae-8cce-4812-a6dd-fd8afdb3e3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:22.975877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:22.995209 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6036c2b-54a7-4bbc-ad61-263707af1616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:22.995879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6036c2b-54a7-4bbc-ad61-263707af1616 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:22.996160 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6036c2b-54a7-4bbc-ad61-263707af1616 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:22.996450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6036c2b-54a7-4bbc-ad61-263707af1616 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:22.997463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6036c2b-54a7-4bbc-ad61-263707af1616 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:22.997978 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6036c2b-54a7-4bbc-ad61-263707af1616 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:22.999554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6036c2b-54a7-4bbc-ad61-263707af1616 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:23.013772 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 452/888] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:23.021544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6036c2b-54a7-4bbc-ad61-263707af1616 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:23.054441 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 437/889] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:23.077112 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f730cae-8cce-4812-a6dd-fd8afdb3e3c7 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:23.080138 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:23.129502 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6036c2b-54a7-4bbc-ad61-263707af1616 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:23.171652 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fdad8-aba0-4869-a3c4-c6a0ab60bb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:23.172830 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fdad8-aba0-4869-a3c4-c6a0ab60bb55 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:23.173268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fdad8-aba0-4869-a3c4-c6a0ab60bb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:23.173833 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fdad8-aba0-4869-a3c4-c6a0ab60bb55 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:23.182908 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9fdad8-aba0-4869-a3c4-c6a0ab60bb55 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:23.184995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9fdad8-aba0-4869-a3c4-c6a0ab60bb55 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:23.195108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f730cae-8cce-4812-a6dd-fd8afdb3e3c7 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:23.203825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-2a9fdad8-aba0-4869-a3c4-c6a0ab60bb55 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 25 15:54:23.203825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:23.203825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:23.203825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:23.203825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:23.203825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:23.203825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:23.203825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:23.203825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:23.203825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:23.203825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:23.203825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:23.203825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:54:23.203825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:54:23.203825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:54:23.203825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:54:23.204912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:23.204912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:23.204912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:54:23.204912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:23.204912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:54:23.204912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:23.204912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:23.204912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:23.204912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:23.204912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:23.204912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:23.204912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:23.204912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:23.204912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:23.204912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:23.204912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:23.204912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:23.204912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:23.206060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:23.206060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:23.206060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:54:23.206060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:54:23.206060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:54:23.206060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:54:23.206060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:54:23.206060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:54:23.206060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 25 15:54:23.206060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:23.208885 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 453/890] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Jun 25 15:54:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:23.214677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fdad8-aba0-4869-a3c4-c6a0ab60bb55 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:23.240549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4acb90-8b6c-4e69-b1a6-81299a6c8e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:23.241888 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4acb90-8b6c-4e69-b1a6-81299a6c8e19 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:23.242295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4acb90-8b6c-4e69-b1a6-81299a6c8e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:23.242678 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4acb90-8b6c-4e69-b1a6-81299a6c8e19 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:23.243760 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4acb90-8b6c-4e69-b1a6-81299a6c8e19 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:23.245698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4acb90-8b6c-4e69-b1a6-81299a6c8e19 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:23.260734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-af4acb90-8b6c-4e69-b1a6-81299a6c8e19 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 25 15:54:23.260734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:23.260734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:23.260734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:23.260734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:23.260734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:23.260734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:23.260734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:23.260734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 25 15:54:23.260734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:23.260734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:23.260734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:23.260734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:23.260734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:23.260734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:23.260734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:23.260734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:23.261772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:23.261772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:23.261772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:23.261772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:23.261772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:23.261772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:23.261772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:23.261772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:23.261772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:23.261772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:23.261772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:23.261772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:23.261772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:23.261772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:23.261772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:23.261772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:23.262632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:23.262632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:23.262632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:23.262632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:23.262632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:23.262632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:23.262632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:23.262632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:23.262632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:23.262632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:23.262632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:23.262632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:23.262632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 25 15:54:23.262632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:23.262632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 438/891] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:54:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:23.268152 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4acb90-8b6c-4e69-b1a6-81299a6c8e19 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:23.290558 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:23.357358 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fdad8-aba0-4869-a3c4-c6a0ab60bb55 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:23.370951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4acb90-8b6c-4e69-b1a6-81299a6c8e19 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:23.374877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:23.403862 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a5b5c-fcb2-4576-89ff-55209317d255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:23.404831 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a5b5c-fcb2-4576-89ff-55209317d255 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:23.405314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a5b5c-fcb2-4576-89ff-55209317d255 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:23.405757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a5b5c-fcb2-4576-89ff-55209317d255 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:23.406945 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a5b5c-fcb2-4576-89ff-55209317d255 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:23.407522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a5b5c-fcb2-4576-89ff-55209317d255 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:23.409239 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a5b5c-fcb2-4576-89ff-55209317d255 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:23.409427 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4276b1-4617-4648-99d4-2c696462f70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:23.410205 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4276b1-4617-4648-99d4-2c696462f70e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:23.410554 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4276b1-4617-4648-99d4-2c696462f70e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:23.410893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4276b1-4617-4648-99d4-2c696462f70e None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:23.411667 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4276b1-4617-4648-99d4-2c696462f70e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:23.412040 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4276b1-4617-4648-99d4-2c696462f70e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:23.413796 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4276b1-4617-4648-99d4-2c696462f70e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:23.423236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 439/892] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:54:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:23.424704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4276b1-4617-4648-99d4-2c696462f70e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:23.430708 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 454/893] 10.222.0.13 () {56 vars in 1149 bytes} [Sat Jun 25 15:54:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:23.440458 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a5b5c-fcb2-4576-89ff-55209317d255 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:23.494264 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:23.538802 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4276b1-4617-4648-99d4-2c696462f70e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:23.569959 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a5b5c-fcb2-4576-89ff-55209317d255 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:23.574387 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7290ffd-90bb-4dfa-b7fc-043d30ab1bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:23.575323 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7290ffd-90bb-4dfa-b7fc-043d30ab1bfe None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:23.575821 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7290ffd-90bb-4dfa-b7fc-043d30ab1bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:23.576288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7290ffd-90bb-4dfa-b7fc-043d30ab1bfe None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:23.577396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7290ffd-90bb-4dfa-b7fc-043d30ab1bfe None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:23.577912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7290ffd-90bb-4dfa-b7fc-043d30ab1bfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:23.579499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7290ffd-90bb-4dfa-b7fc-043d30ab1bfe None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:23.585135 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 440/894] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:54:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:23.593878 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7290ffd-90bb-4dfa-b7fc-043d30ab1bfe None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:23.611698 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260caed-696c-4ed6-9b4b-b45d5425610c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:23.612793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260caed-696c-4ed6-9b4b-b45d5425610c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:23.613289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260caed-696c-4ed6-9b4b-b45d5425610c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:23.613827 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260caed-696c-4ed6-9b4b-b45d5425610c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:23.660469 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:23.698417 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a260caed-696c-4ed6-9b4b-b45d5425610c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:23.700447 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a260caed-696c-4ed6-9b4b-b45d5425610c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:23.724148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7290ffd-90bb-4dfa-b7fc-043d30ab1bfe None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:23.753265 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:23.760161 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805f09a-8c7a-4415-b9c0-ea4de7556bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:23.760839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805f09a-8c7a-4415-b9c0-ea4de7556bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:23.761118 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805f09a-8c7a-4415-b9c0-ea4de7556bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:23.761388 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805f09a-8c7a-4415-b9c0-ea4de7556bc8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:23.762185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805f09a-8c7a-4415-b9c0-ea4de7556bc8 None admin] RBAC: Adding query filter params (interface=public, service_id=25ff654a76d347568da8dcec503196d6, region_id=RegionOne) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:23.762492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805f09a-8c7a-4415-b9c0-ea4de7556bc8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:23.764042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805f09a-8c7a-4415-b9c0-ea4de7556bc8 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:23.769980 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 441/895] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Jun 25 15:54:23 2022] GET /identity/v3/endpoints?service_id=25ff654a76d347568da8dcec503196d6&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:23.777126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805f09a-8c7a-4415-b9c0-ea4de7556bc8 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:23.846566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 455/896] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:54:23 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:54:23.851514 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260caed-696c-4ed6-9b4b-b45d5425610c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:23.897959 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805f09a-8c7a-4415-b9c0-ea4de7556bc8 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:23.935091 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7663a2-35e2-4554-b387-face367a1a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:23.936062 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7663a2-35e2-4554-b387-face367a1a93 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:23.936657 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7663a2-35e2-4554-b387-face367a1a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:23.937099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7663a2-35e2-4554-b387-face367a1a93 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:23.938085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7663a2-35e2-4554-b387-face367a1a93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:23.939622 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7663a2-35e2-4554-b387-face367a1a93 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:23.945822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:23.946590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 442/897] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Jun 25 15:54:23 2022] GET /identity/v3/services => generated 1268 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 15:54:23.980255 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260caed-696c-4ed6-9b4b-b45d5425610c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:24.018323 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfa8bf-9f07-463d-b739-81a07d452c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:24.019150 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfa8bf-9f07-463d-b739-81a07d452c7b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:24.019435 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfa8bf-9f07-463d-b739-81a07d452c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:24.019714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfa8bf-9f07-463d-b739-81a07d452c7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:24.031570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cfa8bf-9f07-463d-b739-81a07d452c7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:24.033075 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cfa8bf-9f07-463d-b739-81a07d452c7b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:24.052450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-99cfa8bf-9f07-463d-b739-81a07d452c7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:24.052450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:24.052450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:24.052450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:24.052450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:24.052450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:24.052450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:24.052450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:24.052450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:24.052450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:24.052450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:24.052450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:24.052450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:24.052450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:24.052450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:24.053530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:24.053530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:24.053530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:24.053530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:24.053530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:24.053530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:24.053530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:24.053530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:24.053530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:24.053530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:24.053530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:24.053530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:24.053530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:24.053530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:24.053530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:24.053530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:24.053530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:24.054440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:24.054440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:24.054440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:24.054440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:24.054440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 456/898] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:24.060005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7663a2-35e2-4554-b387-face367a1a93 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:24.124134 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:24.168820 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7663a2-35e2-4554-b387-face367a1a93 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:24.203240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aeeca4-ecbd-4e6c-9136-41f40f2508e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:24.204260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aeeca4-ecbd-4e6c-9136-41f40f2508e4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:24.204760 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aeeca4-ecbd-4e6c-9136-41f40f2508e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:24.205226 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aeeca4-ecbd-4e6c-9136-41f40f2508e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:24.206137 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aeeca4-ecbd-4e6c-9136-41f40f2508e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:24.206649 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aeeca4-ecbd-4e6c-9136-41f40f2508e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:24.208200 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aeeca4-ecbd-4e6c-9136-41f40f2508e4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:24.221661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 443/899] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:24.228446 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfa8bf-9f07-463d-b739-81a07d452c7b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:24.329996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfa8bf-9f07-463d-b739-81a07d452c7b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:24.353963 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:24.365556 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ea51b-229e-419b-ac5f-67aceba7b4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:24.366642 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ea51b-229e-419b-ac5f-67aceba7b4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:24.367383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ea51b-229e-419b-ac5f-67aceba7b4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:24.367659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ea51b-229e-419b-ac5f-67aceba7b4e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:24.374624 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ea51b-229e-419b-ac5f-67aceba7b4e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:24.376086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ea51b-229e-419b-ac5f-67aceba7b4e1 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:24.390332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-be8ea51b-229e-419b-ac5f-67aceba7b4e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:24.390332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:24.390332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:24.390332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:24.390332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:24.390332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:24.390332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:24.390332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:24.390332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:24.390332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:24.390332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:24.390332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:24.390332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:54:24.390332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:54:24.390332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:54:24.391560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:54:24.391560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:24.391560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:24.391560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:24.391560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:24.391560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:24.391560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:24.391560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:24.391560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:24.391560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:24.391560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:24.391560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:24.391560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:24.391560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:24.391560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:24.391560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:24.391560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:24.392479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:54:24.392479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:54:24.392479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:54:24.392479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:54:24.392479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:54:24.392479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:54:24.392479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:24.392479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:24.392479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 457/900] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:24.395035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d3804-008e-4d4c-9b4d-4ba2c4b2f4fe None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:24.395530 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d3804-008e-4d4c-9b4d-4ba2c4b2f4fe None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:24.396003 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d3804-008e-4d4c-9b4d-4ba2c4b2f4fe None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:24.397654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ea51b-229e-419b-ac5f-67aceba7b4e1 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:24.398028 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 444/901] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:24.403409 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b903d-91f5-4f36-a796-af6874f8c0ef None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:24.403696 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b903d-91f5-4f36-a796-af6874f8c0ef None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:24.403976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b903d-91f5-4f36-a796-af6874f8c0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:24.421505 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-0b6b903d-91f5-4f36-a796-af6874f8c0ef None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:24.422699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-0b6b903d-91f5-4f36-a796-af6874f8c0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:24.504689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 445/902] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:24.515644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ea51b-229e-419b-ac5f-67aceba7b4e1 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:24.528132 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b903d-91f5-4f36-a796-af6874f8c0ef None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:24.561655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2524ef-3d91-4dec-8d16-fdb58d8e4b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:24.563253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2524ef-3d91-4dec-8d16-fdb58d8e4b1c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:24.563885 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2524ef-3d91-4dec-8d16-fdb58d8e4b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:24.564510 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2524ef-3d91-4dec-8d16-fdb58d8e4b1c None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:24.565760 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2524ef-3d91-4dec-8d16-fdb58d8e4b1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:24.566637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2524ef-3d91-4dec-8d16-fdb58d8e4b1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:24.568608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2524ef-3d91-4dec-8d16-fdb58d8e4b1c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:24.590787 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:24.593149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 458/903] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:54:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:24.599141 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2524ef-3d91-4dec-8d16-fdb58d8e4b1c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:24.653629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:24.655067 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b903d-91f5-4f36-a796-af6874f8c0ef None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:24.691145 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298aed71-7618-4ffe-8612-63a5781b3093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:24.692006 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298aed71-7618-4ffe-8612-63a5781b3093 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:24.692288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298aed71-7618-4ffe-8612-63a5781b3093 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:24.692664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298aed71-7618-4ffe-8612-63a5781b3093 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:24.693328 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298aed71-7618-4ffe-8612-63a5781b3093 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:24.694718 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298aed71-7618-4ffe-8612-63a5781b3093 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:24.707196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-298aed71-7618-4ffe-8612-63a5781b3093 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 25 15:54:24.707196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:24.707196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:24.707196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:24.707196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:24.707196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:24.707196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:24.707196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:24.707196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 25 15:54:24.707196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:24.707196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:24.707196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:24.707196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:24.707196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:24.707196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:24.707196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:24.708324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:24.708324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:24.708324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:24.708324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:24.708324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:24.708324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:24.708324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:24.708324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:24.708324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:24.708324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:24.708324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:24.708324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:24.708324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:24.708324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:24.708324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:24.708324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:24.709217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:24.709217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:24.709217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:24.709217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:24.709217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:24.709217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:24.709217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:24.709217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:24.709217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:24.709217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:24.709217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:24.709217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:24.709217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:24.709217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 25 15:54:24.709217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:24.709217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 446/904] 10.222.0.13 () {56 vars in 1132 bytes} [Sat Jun 25 15:54:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:24.710056 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0259afe-c3eb-42d6-9c5d-6780359a46ea None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:24.710368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0259afe-c3eb-42d6-9c5d-6780359a46ea None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:24.710460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0259afe-c3eb-42d6-9c5d-6780359a46ea None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:24.712814 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 447/905] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:24.716449 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0259afe-c3eb-42d6-9c5d-6780359a46ea None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:24.724090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2524ef-3d91-4dec-8d16-fdb58d8e4b1c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:24.762198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836b315-f09f-4f26-9a7b-0f578f2b3752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:24.762770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836b315-f09f-4f26-9a7b-0f578f2b3752 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:24.763054 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836b315-f09f-4f26-9a7b-0f578f2b3752 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:24.763330 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836b315-f09f-4f26-9a7b-0f578f2b3752 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:24.764169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836b315-f09f-4f26-9a7b-0f578f2b3752 None admin] RBAC: Adding query filter params (role.id=c38990a9778e4b2abd3376f9e066d3a5, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=28b3e73439e64050b406a9f3e9b0160a) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:24.764520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836b315-f09f-4f26-9a7b-0f578f2b3752 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:24.765963 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836b315-f09f-4f26-9a7b-0f578f2b3752 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:24.780438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 459/906] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:54:24 2022] GET /identity/v3/role_assignments?user.id=28b3e73439e64050b406a9f3e9b0160a&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=c38990a9778e4b2abd3376f9e066d3a5 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:24.780438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb5b7d-72a4-49dd-9462-8ba1fc50c356 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:24.780438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb5b7d-72a4-49dd-9462-8ba1fc50c356 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:24.780438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb5b7d-72a4-49dd-9462-8ba1fc50c356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:24.799917 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8beb5b7d-72a4-49dd-9462-8ba1fc50c356 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:24.801380 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8beb5b7d-72a4-49dd-9462-8ba1fc50c356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:24.865051 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0259afe-c3eb-42d6-9c5d-6780359a46ea None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:24.882941 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:24.900433 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 460/907] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:24.912378 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9161ddc-aabe-46cd-af76-02dc39ab3116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:24.913094 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9161ddc-aabe-46cd-af76-02dc39ab3116 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:24.913369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9161ddc-aabe-46cd-af76-02dc39ab3116 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:24.913644 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9161ddc-aabe-46cd-af76-02dc39ab3116 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:24.914468 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9161ddc-aabe-46cd-af76-02dc39ab3116 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:24.914779 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9161ddc-aabe-46cd-af76-02dc39ab3116 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:24.916253 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9161ddc-aabe-46cd-af76-02dc39ab3116 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:24.920240 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb5b7d-72a4-49dd-9462-8ba1fc50c356 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:24.930333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 448/908] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Jun 25 15:54:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:24.936686 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9161ddc-aabe-46cd-af76-02dc39ab3116 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:24.960689 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:25.046556 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9161ddc-aabe-46cd-af76-02dc39ab3116 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:25.046736 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb5b7d-72a4-49dd-9462-8ba1fc50c356 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:25.083839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e25828-7e8f-4225-be84-708d0c29b36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:25.084536 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e25828-7e8f-4225-be84-708d0c29b36a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:25.084822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e25828-7e8f-4225-be84-708d0c29b36a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:25.085095 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e25828-7e8f-4225-be84-708d0c29b36a None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:25.085852 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e25828-7e8f-4225-be84-708d0c29b36a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:25.086176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e25828-7e8f-4225-be84-708d0c29b36a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:25.087609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e25828-7e8f-4225-be84-708d0c29b36a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:25.092908 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a942b-6c2f-4cf7-a189-2e5785449b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:25.093862 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a942b-6c2f-4cf7-a189-2e5785449b65 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:25.094387 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a942b-6c2f-4cf7-a189-2e5785449b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:25.094852 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a942b-6c2f-4cf7-a189-2e5785449b65 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:25.095782 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a942b-6c2f-4cf7-a189-2e5785449b65 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:25.097365 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a942b-6c2f-4cf7-a189-2e5785449b65 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:25.097953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 449/909] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Jun 25 15:54:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:25.098885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e25828-7e8f-4225-be84-708d0c29b36a None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:25.116009 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 461/910] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Jun 25 15:54:24 2022] POST /identity/v3/services => generated 241 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 15:54:25.160530 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:25.229827 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e25828-7e8f-4225-be84-708d0c29b36a None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:25.277813 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f7b6b-72ef-4310-871b-8759f839cad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:25.278471 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f7b6b-72ef-4310-871b-8759f839cad0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:25.278749 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f7b6b-72ef-4310-871b-8759f839cad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:25.279016 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f7b6b-72ef-4310-871b-8759f839cad0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:25.279796 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f7b6b-72ef-4310-871b-8759f839cad0 None admin] RBAC: Adding query filter params (interface=public, service_id=ee60de0f447143c49df3ec8f424a1ce6, region_id=RegionOne) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:25.280097 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f7b6b-72ef-4310-871b-8759f839cad0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:25.281496 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f7b6b-72ef-4310-871b-8759f839cad0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:25.287105 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 450/911] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Jun 25 15:54:25 2022] GET /identity/v3/endpoints?service_id=ee60de0f447143c49df3ec8f424a1ce6&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:25.292732 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a942b-6c2f-4cf7-a189-2e5785449b65 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:25.397450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:25.404076 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a942b-6c2f-4cf7-a189-2e5785449b65 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:25.439234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab8ebb-9205-4ce4-96af-4588fa1a0308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:25.439904 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab8ebb-9205-4ce4-96af-4588fa1a0308 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:25.440186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab8ebb-9205-4ce4-96af-4588fa1a0308 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:25.440488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab8ebb-9205-4ce4-96af-4588fa1a0308 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:25.441173 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab8ebb-9205-4ce4-96af-4588fa1a0308 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:25.442537 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab8ebb-9205-4ce4-96af-4588fa1a0308 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:25.449199 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 462/912] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Jun 25 15:54:25 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 15:54:26.059471 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685e8b2-43e3-4309-8b0b-f7fd06e1f819 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:26.060100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685e8b2-43e3-4309-8b0b-f7fd06e1f819 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:26.060100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685e8b2-43e3-4309-8b0b-f7fd06e1f819 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:26.061666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 451/913] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:26.067060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c943d4b3-b4de-48fd-9837-79e72a397033 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:26.067650 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c943d4b3-b4de-48fd-9837-79e72a397033 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:26.067650 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c943d4b3-b4de-48fd-9837-79e72a397033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:26.084551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c943d4b3-b4de-48fd-9837-79e72a397033 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:26.085783 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c943d4b3-b4de-48fd-9837-79e72a397033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:26.156597 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 463/914] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:26.174988 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685e8b2-43e3-4309-8b0b-f7fd06e1f819 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:26.281794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:26.286726 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685e8b2-43e3-4309-8b0b-f7fd06e1f819 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:26.322279 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83074f-1a8c-45b9-930e-7664ae501675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:26.323083 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83074f-1a8c-45b9-930e-7664ae501675 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:26.323359 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83074f-1a8c-45b9-930e-7664ae501675 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:26.323630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83074f-1a8c-45b9-930e-7664ae501675 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:26.324285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83074f-1a8c-45b9-930e-7664ae501675 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:26.325700 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83074f-1a8c-45b9-930e-7664ae501675 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:26.337695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-ba83074f-1a8c-45b9-930e-7664ae501675 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 25 15:54:26.337695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:26.337695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:26.337695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:26.337695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:26.337695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:26.337695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:26.337695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:26.337695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 25 15:54:26.337695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:26.337695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:26.337695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:26.337695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:26.337695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:26.337695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:26.337695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:26.337695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:26.338756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:26.338756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:26.338756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:26.338756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:26.338756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:26.338756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:26.338756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:26.338756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:26.338756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:26.338756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:26.338756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:26.338756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:26.338756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:26.338756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:26.338756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:26.338756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:26.339671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:26.339671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:26.339671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:26.339671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:26.339671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:26.339671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:26.339671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:26.339671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:26.339671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:26.339671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:26.339671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:26.339671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:26.339671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 25 15:54:26.339671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:26.339671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 452/915] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:54:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:26.344996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c943d4b3-b4de-48fd-9837-79e72a397033 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:26.440023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:26.449903 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c943d4b3-b4de-48fd-9837-79e72a397033 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:26.484369 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22bab1-b9dd-401e-9b3b-a6eeaaf809b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:26.485060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22bab1-b9dd-401e-9b3b-a6eeaaf809b4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:26.485340 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22bab1-b9dd-401e-9b3b-a6eeaaf809b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:26.485620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22bab1-b9dd-401e-9b3b-a6eeaaf809b4 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:26.486325 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22bab1-b9dd-401e-9b3b-a6eeaaf809b4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:26.486629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22bab1-b9dd-401e-9b3b-a6eeaaf809b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:26.487992 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22bab1-b9dd-401e-9b3b-a6eeaaf809b4 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:26.493295 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 464/916] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:54:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:26.498710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83074f-1a8c-45b9-930e-7664ae501675 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:26.608581 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83074f-1a8c-45b9-930e-7664ae501675 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:26.645451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14700ab-08b7-4873-90ed-650eaef392b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:26.646124 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14700ab-08b7-4873-90ed-650eaef392b1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:26.646402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14700ab-08b7-4873-90ed-650eaef392b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:26.646673 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14700ab-08b7-4873-90ed-650eaef392b1 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:26.647390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14700ab-08b7-4873-90ed-650eaef392b1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:26.647692 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14700ab-08b7-4873-90ed-650eaef392b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:26.649128 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14700ab-08b7-4873-90ed-650eaef392b1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:26.654834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 453/917] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Jun 25 15:54:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:26.660189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22bab1-b9dd-401e-9b3b-a6eeaaf809b4 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:26.769134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22bab1-b9dd-401e-9b3b-a6eeaaf809b4 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:26.806678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5709f7-0df9-40ee-a0cf-e6f5a0a0f49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:26.807345 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5709f7-0df9-40ee-a0cf-e6f5a0a0f49d None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:26.807625 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5709f7-0df9-40ee-a0cf-e6f5a0a0f49d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:26.807898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5709f7-0df9-40ee-a0cf-e6f5a0a0f49d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:26.808751 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5709f7-0df9-40ee-a0cf-e6f5a0a0f49d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:26.810115 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5709f7-0df9-40ee-a0cf-e6f5a0a0f49d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:26.832844 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 465/918] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Jun 25 15:54:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:54:27.020522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38ad7a-55ca-48aa-97e1-8f3b3e63b555 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:27.021045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38ad7a-55ca-48aa-97e1-8f3b3e63b555 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:27.021321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38ad7a-55ca-48aa-97e1-8f3b3e63b555 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:27.022967 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 454/919] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:27.029135 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51dfdd-5c5a-405c-8512-bd250ceca5d7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:27.029434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51dfdd-5c5a-405c-8512-bd250ceca5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:27.029696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51dfdd-5c5a-405c-8512-bd250ceca5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:27.050144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3a51dfdd-5c5a-405c-8512-bd250ceca5d7 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:27.051343 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3a51dfdd-5c5a-405c-8512-bd250ceca5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:27.054599 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:27.146130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 466/920] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:27.169738 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38ad7a-55ca-48aa-97e1-8f3b3e63b555 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:27.201778 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:27.281262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38ad7a-55ca-48aa-97e1-8f3b3e63b555 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:27.296967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3bd7b-932a-4773-978c-d7f0c95a40ba None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:27.297317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3bd7b-932a-4773-978c-d7f0c95a40ba None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:27.297520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3bd7b-932a-4773-978c-d7f0c95a40ba None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:27.299186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 467/921] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:27.304475 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd58ed4-e961-4b63-a2b0-58d8d9de07d9 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:27.304792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd58ed4-e961-4b63-a2b0-58d8d9de07d9 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:27.305074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd58ed4-e961-4b63-a2b0-58d8d9de07d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:27.325402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5e273-a468-4338-8d38-cc4ea9da1f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:27.326106 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5dd58ed4-e961-4b63-a2b0-58d8d9de07d9 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:27.326488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5e273-a468-4338-8d38-cc4ea9da1f2b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:27.326982 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5e273-a468-4338-8d38-cc4ea9da1f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:27.327325 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5dd58ed4-e961-4b63-a2b0-58d8d9de07d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:27.327432 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5e273-a468-4338-8d38-cc4ea9da1f2b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:27.344112 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5e273-a468-4338-8d38-cc4ea9da1f2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:54:27.353928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-6ac5e273-a468-4338-8d38-cc4ea9da1f2b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 25 15:54:27.353928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:27.353928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:27.353928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:27.353928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:27.353928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:27.353928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:27.353928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:27.353928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:27.353928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:27.353928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:27.353928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:27.353928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:54:27.353928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:54:27.353928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:54:27.353928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:54:27.354943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:54:27.354943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:54:27.354943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:54:27.354943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:54:27.354943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:27.354943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:27.354943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:27.354943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:27.354943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:27.354943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:27.354943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:27.354943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:27.354943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:27.354943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:27.354943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:27.354943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:27.355983 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:27.355983 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:27.355983 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:27.355983 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:27.355983 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:54:27.355983 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:54:27.355983 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:54:27.355983 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:54:27.355983 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:54:27.355983 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:54:27.355983 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 25 15:54:27.355983 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:27.355983 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 455/922] 10.222.0.13 () {56 vars in 1120 bytes} [Sat Jun 25 15:54:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:27.363369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5e273-a468-4338-8d38-cc4ea9da1f2b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:27.426825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 468/923] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:27.459222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd58ed4-e961-4b63-a2b0-58d8d9de07d9 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:27.494348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:27.514327 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5e273-a468-4338-8d38-cc4ea9da1f2b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:27.556093 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb29b8d-9148-4376-97e8-b351e13ab92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:27.556815 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:27.557358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb29b8d-9148-4376-97e8-b351e13ab92b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:27.559571 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb29b8d-9148-4376-97e8-b351e13ab92b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:27.560079 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb29b8d-9148-4376-97e8-b351e13ab92b None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:27.561117 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb29b8d-9148-4376-97e8-b351e13ab92b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:27.561716 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb29b8d-9148-4376-97e8-b351e13ab92b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:27.563291 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb29b8d-9148-4376-97e8-b351e13ab92b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:27.574566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 456/924] 10.222.0.13 () {56 vars in 1115 bytes} [Sat Jun 25 15:54:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:27.580753 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb29b8d-9148-4376-97e8-b351e13ab92b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:27.610370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd58ed4-e961-4b63-a2b0-58d8d9de07d9 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:27.652698 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c51bc-d424-468b-b064-c407fb3a813f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:27.653550 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c51bc-d424-468b-b064-c407fb3a813f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:27.653829 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c51bc-d424-468b-b064-c407fb3a813f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:27.654114 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c51bc-d424-468b-b064-c407fb3a813f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:27.654967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c51bc-d424-468b-b064-c407fb3a813f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:27.656582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c51bc-d424-468b-b064-c407fb3a813f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:27.671090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-0e3c51bc-d424-468b-b064-c407fb3a813f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 25 15:54:27.671090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:27.671090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:27.671090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:27.671090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:27.671090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:27.671090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:27.671090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:27.671090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 25 15:54:27.671090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:27.671090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:27.671090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:27.671090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:27.671090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:27.671090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:27.671090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:27.671090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:27.672196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:27.672196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:27.672196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:27.672196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:27.672196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:27.672196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:27.672196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:27.672196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:27.672196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:27.672196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:27.672196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:27.672196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:27.672196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:27.672196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:27.672196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:27.672196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:27.673080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:27.673080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:27.673080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:27.673080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:27.673080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:27.673080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:27.673080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:27.673080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:27.673080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:27.673080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:27.673080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:27.673080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:27.673080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 25 15:54:27.673080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:27.677471 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 469/925] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Jun 25 15:54:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:27.678476 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c51bc-d424-468b-b064-c407fb3a813f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:27.709480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb29b8d-9148-4376-97e8-b351e13ab92b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:27.746278 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ae280-92d4-4809-bd36-04e53799ff75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:27.747082 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ae280-92d4-4809-bd36-04e53799ff75 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:27.747362 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ae280-92d4-4809-bd36-04e53799ff75 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:27.747631 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ae280-92d4-4809-bd36-04e53799ff75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:27.758806 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ae280-92d4-4809-bd36-04e53799ff75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:27.760255 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ae280-92d4-4809-bd36-04e53799ff75 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:27.778230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-901ae280-92d4-4809-bd36-04e53799ff75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:27.778230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:27.778230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:27.778230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:27.778230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:27.778230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:27.778230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:27.778230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:27.778230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:27.778230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:27.778230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:27.778230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:27.778230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:27.778230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:27.778230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:27.779334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:27.779334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:27.779334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:27.779334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:27.779334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:27.779334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:27.779334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:27.779334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:27.779334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:27.779334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:27.779334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:27.779334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:27.779334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:27.779334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:27.779334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:27.779334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:27.779334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:27.780495 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:27.780495 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:27.780495 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:27.780495 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:27.784147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 457/926] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:27.785458 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93023e9-5c9b-4796-b690-4e2bbbfcc020 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:27.785741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93023e9-5c9b-4796-b690-4e2bbbfcc020 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:27.786015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93023e9-5c9b-4796-b690-4e2bbbfcc020 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:27.786383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c51bc-d424-468b-b064-c407fb3a813f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:27.787399 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 458/927] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:27.788036 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93023e9-5c9b-4796-b690-4e2bbbfcc020 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:27.791006 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:27.821784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:27.829485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339fad5-8653-43f4-9ad9-562259f1e0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:27.830150 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339fad5-8653-43f4-9ad9-562259f1e0cc None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:27.830429 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339fad5-8653-43f4-9ad9-562259f1e0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:27.830700 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339fad5-8653-43f4-9ad9-562259f1e0cc None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:27.831403 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339fad5-8653-43f4-9ad9-562259f1e0cc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:27.831708 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339fad5-8653-43f4-9ad9-562259f1e0cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:27.833518 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339fad5-8653-43f4-9ad9-562259f1e0cc None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:27.846661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 470/928] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Jun 25 15:54:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:27.846661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418dd02-5ddc-46d3-854b-12a912bfb08e None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:27.846661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418dd02-5ddc-46d3-854b-12a912bfb08e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:27.846661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418dd02-5ddc-46d3-854b-12a912bfb08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:27.866317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8418dd02-5ddc-46d3-854b-12a912bfb08e None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:27.867718 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8418dd02-5ddc-46d3-854b-12a912bfb08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:27.900578 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93023e9-5c9b-4796-b690-4e2bbbfcc020 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:27.935050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d269308-d605-45df-b555-3d19c68c7d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:27.935711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d269308-d605-45df-b555-3d19c68c7d73 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:27.935991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d269308-d605-45df-b555-3d19c68c7d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:27.936266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d269308-d605-45df-b555-3d19c68c7d73 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:27.937017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d269308-d605-45df-b555-3d19c68c7d73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:27.937323 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d269308-d605-45df-b555-3d19c68c7d73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:27.948622 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 471/929] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:27.948822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d269308-d605-45df-b555-3d19c68c7d73 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:27.948923 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418dd02-5ddc-46d3-854b-12a912bfb08e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:27.951447 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 459/930] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:27.957060 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d269308-d605-45df-b555-3d19c68c7d73 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:28.020390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:28.059899 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418dd02-5ddc-46d3-854b-12a912bfb08e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:28.070240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d269308-d605-45df-b555-3d19c68c7d73 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:28.086939 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:28.099716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a2e0ec-1b5b-4379-8a23-b47a305ce47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:28.100389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2e0ec-1b5b-4379-8a23-b47a305ce47d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:28.100698 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2e0ec-1b5b-4379-8a23-b47a305ce47d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:28.100972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2e0ec-1b5b-4379-8a23-b47a305ce47d None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:28.101775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a2e0ec-1b5b-4379-8a23-b47a305ce47d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:28.102081 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a2e0ec-1b5b-4379-8a23-b47a305ce47d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:28.103452 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a2e0ec-1b5b-4379-8a23-b47a305ce47d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:28.108566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 472/931] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Jun 25 15:54:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:28.109331 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a2e0ec-1b5b-4379-8a23-b47a305ce47d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:28.115186 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d1612-ac46-4090-902c-57ba59753d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:28.116127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d1612-ac46-4090-902c-57ba59753d87 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:28.116498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d1612-ac46-4090-902c-57ba59753d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:28.116873 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d1612-ac46-4090-902c-57ba59753d87 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:28.126387 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d1612-ac46-4090-902c-57ba59753d87 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:28.128758 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d1612-ac46-4090-902c-57ba59753d87 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:28.149342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-c33d1612-ac46-4090-902c-57ba59753d87 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 25 15:54:28.149342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:28.149342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:28.149342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:28.149342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:28.149342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:28.149342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:28.149342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:28.149342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:28.149342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:28.149342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:28.149342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:28.149342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:54:28.149342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:54:28.149342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:54:28.149342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:54:28.150391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:28.150391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:28.150391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:54:28.150391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:28.150391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:54:28.150391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:28.150391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:28.150391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:28.150391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:28.150391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:28.150391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:28.150391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:28.150391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:28.150391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:28.150391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:28.150391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:28.150391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:28.150391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:28.151348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:28.151348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:28.151348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:54:28.151348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:54:28.151348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:54:28.151348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:54:28.151348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:54:28.151348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:54:28.151348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 25 15:54:28.151348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:28.151348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 460/932] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Jun 25 15:54:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:28.151973 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d1612-ac46-4090-902c-57ba59753d87 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:28.218388 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a2e0ec-1b5b-4379-8a23-b47a305ce47d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:28.253514 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf31fa-e0c3-43ea-b6f2-ab555825edee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:28.254603 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf31fa-e0c3-43ea-b6f2-ab555825edee None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:28.255068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf31fa-e0c3-43ea-b6f2-ab555825edee None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:28.255512 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf31fa-e0c3-43ea-b6f2-ab555825edee None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:28.255911 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d1612-ac46-4090-902c-57ba59753d87 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:28.256879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf31fa-e0c3-43ea-b6f2-ab555825edee None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:28.258509 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf31fa-e0c3-43ea-b6f2-ab555825edee None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:28.270952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-4abf31fa-e0c3-43ea-b6f2-ab555825edee None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 25 15:54:28.270952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:28.270952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:28.270952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:28.270952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:28.270952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:28.270952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:28.270952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:28.270952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 25 15:54:28.270952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:28.270952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:28.270952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:28.270952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:28.270952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:28.270952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:28.270952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:28.272121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:28.272121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:28.272121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:28.272121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:28.272121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:28.272121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:28.272121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:28.272121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:28.272121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:28.272121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:28.272121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:28.272121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:28.272121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:28.272121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:28.272121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:28.272121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:28.273414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:28.273414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:28.273414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:28.273414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:28.273414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:28.273414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:28.273414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:28.273414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:28.273414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:28.273414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:28.273414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:28.273414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:28.273414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:28.273414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 25 15:54:28.273414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:28.275625 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 473/933] 10.222.0.13 () {56 vars in 1132 bytes} [Sat Jun 25 15:54:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:28.276604 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf31fa-e0c3-43ea-b6f2-ab555825edee None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:28.285144 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:28.296316 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773e00f-3104-4cf3-a42b-e37cb68b85a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:28.297016 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773e00f-3104-4cf3-a42b-e37cb68b85a5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:28.297297 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773e00f-3104-4cf3-a42b-e37cb68b85a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:28.297570 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773e00f-3104-4cf3-a42b-e37cb68b85a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:28.298354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773e00f-3104-4cf3-a42b-e37cb68b85a5 None admin] RBAC: Adding query filter params (interface=public, service_id=74b93a1263f442ec9fb104dcd03c0463, region_id=RegionOne) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:28.298663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773e00f-3104-4cf3-a42b-e37cb68b85a5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:28.300035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773e00f-3104-4cf3-a42b-e37cb68b85a5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:28.305570 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 461/934] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Jun 25 15:54:28 2022] GET /identity/v3/endpoints?service_id=74b93a1263f442ec9fb104dcd03c0463&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:28.306394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773e00f-3104-4cf3-a42b-e37cb68b85a5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:28.340484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:28.387772 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf31fa-e0c3-43ea-b6f2-ab555825edee None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:28.414293 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773e00f-3104-4cf3-a42b-e37cb68b85a5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:28.422586 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67132c0-7237-47a7-871e-65f06cc64957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:28.423249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67132c0-7237-47a7-871e-65f06cc64957 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:28.423535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67132c0-7237-47a7-871e-65f06cc64957 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:28.423804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67132c0-7237-47a7-871e-65f06cc64957 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:28.424682 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67132c0-7237-47a7-871e-65f06cc64957 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:28.424983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67132c0-7237-47a7-871e-65f06cc64957 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:28.426411 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67132c0-7237-47a7-871e-65f06cc64957 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:28.448217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 474/935] 10.222.0.13 () {56 vars in 1149 bytes} [Sat Jun 25 15:54:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:28.448217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67132c0-7237-47a7-871e-65f06cc64957 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:28.453748 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2cc67-adab-405f-8f2d-26727955653b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:28.454418 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2cc67-adab-405f-8f2d-26727955653b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:28.454695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2cc67-adab-405f-8f2d-26727955653b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:28.454977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2cc67-adab-405f-8f2d-26727955653b None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:28.455668 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2cc67-adab-405f-8f2d-26727955653b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:28.455967 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2cc67-adab-405f-8f2d-26727955653b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:28.457360 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2cc67-adab-405f-8f2d-26727955653b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:28.466943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 462/936] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Jun 25 15:54:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:28.467887 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2cc67-adab-405f-8f2d-26727955653b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:28.519895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:28.549042 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67132c0-7237-47a7-871e-65f06cc64957 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:28.560387 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:28.574401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2cc67-adab-405f-8f2d-26727955653b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:28.586383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686fb6e-d5a1-4559-98c2-9c7da6720c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:28.587051 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686fb6e-d5a1-4559-98c2-9c7da6720c08 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:28.587333 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686fb6e-d5a1-4559-98c2-9c7da6720c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:28.587661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686fb6e-d5a1-4559-98c2-9c7da6720c08 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:28.588351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686fb6e-d5a1-4559-98c2-9c7da6720c08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:28.589747 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686fb6e-d5a1-4559-98c2-9c7da6720c08 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:28.604524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 475/937] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Jun 25 15:54:28 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 15:54:28.604524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686fb6e-d5a1-4559-98c2-9c7da6720c08 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:28.620175 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbb51d-80e5-4025-8f9c-7962abba4a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:28.621163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbb51d-80e5-4025-8f9c-7962abba4a15 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:28.621440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbb51d-80e5-4025-8f9c-7962abba4a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:28.621759 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbb51d-80e5-4025-8f9c-7962abba4a15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:28.634309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbbb51d-80e5-4025-8f9c-7962abba4a15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:28.635904 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbbb51d-80e5-4025-8f9c-7962abba4a15 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:28.656543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-6bbbb51d-80e5-4025-8f9c-7962abba4a15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:28.656543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:28.656543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:28.656543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:28.656543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:28.656543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:28.656543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:28.656543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:28.656543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:28.656543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:28.656543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:28.656543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:28.656543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:28.656543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:28.656543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:28.657694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:28.657694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:28.657694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:28.657694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:28.657694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:28.657694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:28.657694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:28.657694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:28.657694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:28.657694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:28.657694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:28.657694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:28.657694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:28.657694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:28.657694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:28.657694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:28.657694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:28.658810 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:28.658810 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:28.658810 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:28.658810 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:28.662795 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 463/938] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:28.663803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbb51d-80e5-4025-8f9c-7962abba4a15 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:28.734482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686fb6e-d5a1-4559-98c2-9c7da6720c08 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:28.780192 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3a70f-bfce-472a-9ab4-9161c77317c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:28.780998 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3a70f-bfce-472a-9ab4-9161c77317c5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:28.781286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3a70f-bfce-472a-9ab4-9161c77317c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:28.781565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3a70f-bfce-472a-9ab4-9161c77317c5 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:28.782279 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3a70f-bfce-472a-9ab4-9161c77317c5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:28.782588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3a70f-bfce-472a-9ab4-9161c77317c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:28.784006 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3a70f-bfce-472a-9ab4-9161c77317c5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:28.789946 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 476/939] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Jun 25 15:54:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:28.793578 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbb51d-80e5-4025-8f9c-7962abba4a15 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:28.795223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:28.795432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3a70f-bfce-472a-9ab4-9161c77317c5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:28.833820 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4ae99-3bc0-4519-861a-4fed557b0b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:28.834931 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4ae99-3bc0-4519-861a-4fed557b0b37 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:28.835217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4ae99-3bc0-4519-861a-4fed557b0b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:28.835494 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4ae99-3bc0-4519-861a-4fed557b0b37 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:28.836584 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4ae99-3bc0-4519-861a-4fed557b0b37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:28.836893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4ae99-3bc0-4519-861a-4fed557b0b37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:28.840209 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4ae99-3bc0-4519-861a-4fed557b0b37 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:28.850650 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:28.857797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 464/940] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:28.864976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4ae99-3bc0-4519-861a-4fed557b0b37 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:28.934258 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3a70f-bfce-472a-9ab4-9161c77317c5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:28.979758 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8324d-3f96-46e2-aec5-1b364626a0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:28.980945 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8324d-3f96-46e2-aec5-1b364626a0fa None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:28.981618 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8324d-3f96-46e2-aec5-1b364626a0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:28.982035 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8324d-3f96-46e2-aec5-1b364626a0fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:28.983260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f8324d-3f96-46e2-aec5-1b364626a0fa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:28.984657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f8324d-3f96-46e2-aec5-1b364626a0fa None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:28.992257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4ae99-3bc0-4519-861a-4fed557b0b37 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:29.008460 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 477/941] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Jun 25 15:54:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 15:54:29.008460 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94148f5-3f8a-461c-9800-3f25361da2d6 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:29.008460 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94148f5-3f8a-461c-9800-3f25361da2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:29.008813 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94148f5-3f8a-461c-9800-3f25361da2d6 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:29.010601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 478/942] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:29.022615 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be21b7-60e4-479d-8261-5539626c7237 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:29.023004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be21b7-60e4-479d-8261-5539626c7237 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:29.023530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be21b7-60e4-479d-8261-5539626c7237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:29.041859 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a3be21b7-60e4-479d-8261-5539626c7237 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:29.043200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a3be21b7-60e4-479d-8261-5539626c7237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:29.048793 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b678bb09-a56c-421e-90aa-d5452cfcd083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:29.049913 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678bb09-a56c-421e-90aa-d5452cfcd083 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:29.050341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678bb09-a56c-421e-90aa-d5452cfcd083 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:29.050766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678bb09-a56c-421e-90aa-d5452cfcd083 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:29.061969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b678bb09-a56c-421e-90aa-d5452cfcd083 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:29.064349 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b678bb09-a56c-421e-90aa-d5452cfcd083 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:29.087729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-b678bb09-a56c-421e-90aa-d5452cfcd083 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:29.087729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:29.087729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:29.087729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:29.087729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:29.087729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:29.087729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:29.087729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:29.087729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:29.087729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:29.087729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:29.087729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:29.087729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:54:29.087729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:54:29.087729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:54:29.088786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:54:29.088786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:29.088786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:29.088786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:29.088786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:29.088786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:29.088786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:29.088786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:29.088786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:29.088786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:29.088786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:29.088786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:29.088786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:29.088786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:29.088786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:29.088786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:29.088786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:29.089691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:54:29.089691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:54:29.089691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:54:29.089691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:54:29.089691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:54:29.089691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:54:29.089691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:29.089691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:29.089691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 465/943] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:29.100470 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b678bb09-a56c-421e-90aa-d5452cfcd083 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:29.126828 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:29.136461 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 479/944] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:29.154836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be21b7-60e4-479d-8261-5539626c7237 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:29.187014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:29.241511 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b678bb09-a56c-421e-90aa-d5452cfcd083 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:29.278293 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be21b7-60e4-479d-8261-5539626c7237 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:29.287139 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d8deb-e8fa-4eb0-b46c-9bdd202c4585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:29.289043 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d8deb-e8fa-4eb0-b46c-9bdd202c4585 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:29.289549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d8deb-e8fa-4eb0-b46c-9bdd202c4585 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:29.290477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d8deb-e8fa-4eb0-b46c-9bdd202c4585 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:29.291878 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d8deb-e8fa-4eb0-b46c-9bdd202c4585 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:29.292699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d8deb-e8fa-4eb0-b46c-9bdd202c4585 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:29.294623 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d8deb-e8fa-4eb0-b46c-9bdd202c4585 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:29.312881 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 466/945] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:54:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:29.318247 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d8deb-e8fa-4eb0-b46c-9bdd202c4585 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:29.326443 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c97de2-4ddb-400c-81f8-e768ad8507ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:29.328716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c97de2-4ddb-400c-81f8-e768ad8507ec None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:29.329215 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c97de2-4ddb-400c-81f8-e768ad8507ec None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:29.329702 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c97de2-4ddb-400c-81f8-e768ad8507ec None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:29.330629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c97de2-4ddb-400c-81f8-e768ad8507ec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:29.332798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c97de2-4ddb-400c-81f8-e768ad8507ec None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:29.334830 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: WARNING keystone.server.flask.common [None req-34c97de2-4ddb-400c-81f8-e768ad8507ec None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 25 15:54:29.358374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.sql.core [None req-34c97de2-4ddb-400c-81f8-e768ad8507ec None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 25 15:54:29.358374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 25 15:54:29.358374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [parameters: {'id': '28e0b5af8b294d58a96405b3d41a097d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 25 15:54:29.358374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80124) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 25 15:54:29.380803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-34c97de2-4ddb-400c-81f8-e768ad8507ec None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 28e0b5af8b294d58a96405b3d41a097d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 28e0b5af8b294d58a96405b3d41a097d. Jun 25 15:54:29.380803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:29.380803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 25 15:54:29.380803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 25 15:54:29.380803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 25 15:54:29.380803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 25 15:54:29.380803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 25 15:54:29.380803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application result = self._query(query) Jun 25 15:54:29.380803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 25 15:54:29.380803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application conn.query(q) Jun 25 15:54:29.380803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 25 15:54:29.380803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 25 15:54:29.380803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 25 15:54:29.381660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application result.read() Jun 25 15:54:29.381660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 25 15:54:29.381660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 25 15:54:29.381660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 25 15:54:29.381660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 25 15:54:29.381660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 25 15:54:29.381660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 25 15:54:29.381660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 25 15:54:29.381660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 25 15:54:29.381660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 25 15:54:29.381660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:29.381660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 25 15:54:29.381660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:29.381660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:29.381660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 25 15:54:29.381660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 25 15:54:29.381660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 25 15:54:29.381660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 25 15:54:29.382672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 25 15:54:29.382672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application next(self.gen) Jun 25 15:54:29.382672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 25 15:54:29.382672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application yield resource Jun 25 15:54:29.382672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 25 15:54:29.382672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application next(self.gen) Jun 25 15:54:29.382672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 25 15:54:29.382672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self.session.rollback() Jun 25 15:54:29.382672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 25 15:54:29.382672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self.force_reraise() Jun 25 15:54:29.382672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 25 15:54:29.382672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise self.value Jun 25 15:54:29.382672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 25 15:54:29.382672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 25 15:54:29.382672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 25 15:54:29.382672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application session.commit() Jun 25 15:54:29.382672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 25 15:54:29.383568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 25 15:54:29.383568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 25 15:54:29.383568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._prepare_impl() Jun 25 15:54:29.383568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 25 15:54:29.383568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self.session.flush() Jun 25 15:54:29.383568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 25 15:54:29.383568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._flush(objects) Jun 25 15:54:29.383568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 25 15:54:29.383568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 25 15:54:29.383568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 25 15:54:29.383568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application compat.raise_( Jun 25 15:54:29.383568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 25 15:54:29.383568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception Jun 25 15:54:29.383568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 25 15:54:29.383568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application flush_context.execute() Jun 25 15:54:29.383568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 25 15:54:29.383568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rec.execute(self) Jun 25 15:54:29.384578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 25 15:54:29.384578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 25 15:54:29.384578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 25 15:54:29.384578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 25 15:54:29.384578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 25 15:54:29.384578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 25 15:54:29.384578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 25 15:54:29.384578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 25 15:54:29.384578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 25 15:54:29.384578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 25 15:54:29.384578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 25 15:54:29.384578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 25 15:54:29.384578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 25 15:54:29.384578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 25 15:54:29.384578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 25 15:54:29.385482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 25 15:54:29.385482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 25 15:54:29.385482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception Jun 25 15:54:29.385482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 25 15:54:29.385482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 25 15:54:29.385482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 25 15:54:29.385482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 25 15:54:29.385482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 25 15:54:29.385482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application result = self._query(query) Jun 25 15:54:29.385482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 25 15:54:29.385482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application conn.query(q) Jun 25 15:54:29.385482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 25 15:54:29.385482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 25 15:54:29.385482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 25 15:54:29.385482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application result.read() Jun 25 15:54:29.385482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 25 15:54:29.385482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 25 15:54:29.386535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 25 15:54:29.386535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 25 15:54:29.386535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 25 15:54:29.386535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 25 15:54:29.386535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 25 15:54:29.386535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 25 15:54:29.386535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 25 15:54:29.386535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 25 15:54:29.386535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [parameters: {'id': '28e0b5af8b294d58a96405b3d41a097d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 25 15:54:29.386535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 25 15:54:29.386535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:29.386535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:29.386535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:29.386535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:29.386535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 25 15:54:29.387513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 25 15:54:29.387513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 25 15:54:29.387513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 25 15:54:29.387513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 25 15:54:29.387513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:29.387513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:29.387513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:29.387513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:29.387513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:29.387513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:29.387513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:29.387513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:29.387513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:29.387513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:29.387513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:29.387513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:29.387513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:29.388630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:29.388630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 25 15:54:29.388630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 25 15:54:29.388630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:29.388630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:29.388630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 25 15:54:29.388630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 25 15:54:29.388630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 28e0b5af8b294d58a96405b3d41a097d. Jun 25 15:54:29.388630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:29.390811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 480/946] 10.222.0.13 () {60 vars in 1140 bytes} [Sat Jun 25 15:54:29 2022] POST /identity/v3/projects => generated 262 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:29.398390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c97de2-4ddb-400c-81f8-e768ad8507ec None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:29.421135 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:29.446128 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d8deb-e8fa-4eb0-b46c-9bdd202c4585 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:29.483826 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bc40a-9ec2-4d5c-928e-64669211474f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:29.485112 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bc40a-9ec2-4d5c-928e-64669211474f None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:29.485602 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bc40a-9ec2-4d5c-928e-64669211474f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:29.486126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2bc40a-9ec2-4d5c-928e-64669211474f None admin] PATH_INFO: `/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/28b3e73439e64050b406a9f3e9b0160a/roles/c38990a9778e4b2abd3376f9e066d3a5` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:29.488808 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bc40a-9ec2-4d5c-928e-64669211474f None admin] RBAC: Authorizing `identity:create_grant(project_id=3600648eb94b434b8c797d2f454e46b8, user_id=28b3e73439e64050b406a9f3e9b0160a, role_id=c38990a9778e4b2abd3376f9e066d3a5)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:29.490607 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2bc40a-9ec2-4d5c-928e-64669211474f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:29.507188 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 467/947] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:54:29 2022] PUT /identity/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/28b3e73439e64050b406a9f3e9b0160a/roles/c38990a9778e4b2abd3376f9e066d3a5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 15:54:29.514533 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:29.547765 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c97de2-4ddb-400c-81f8-e768ad8507ec None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:29.595517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13101cb1-ad24-4b96-b5a1-e22e427f385c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:29.596762 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13101cb1-ad24-4b96-b5a1-e22e427f385c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:29.597274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13101cb1-ad24-4b96-b5a1-e22e427f385c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:29.597773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13101cb1-ad24-4b96-b5a1-e22e427f385c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:29.606170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13101cb1-ad24-4b96-b5a1-e22e427f385c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:29.608460 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13101cb1-ad24-4b96-b5a1-e22e427f385c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:29.624804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-13101cb1-ad24-4b96-b5a1-e22e427f385c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:29.624804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:29.624804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:29.624804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:29.624804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:29.624804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:29.624804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:29.624804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:29.624804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:29.624804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:29.624804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:29.624804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:29.624804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:54:29.624804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:54:29.624804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:54:29.625818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:54:29.625818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:29.625818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:29.625818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:29.625818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:29.625818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:29.625818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:29.625818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:29.625818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:29.625818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:29.625818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:29.625818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:29.625818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:29.625818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:29.625818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:29.625818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:29.625818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:29.626727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:54:29.626727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:54:29.626727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:54:29.626727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:54:29.626727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:54:29.626727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:54:29.626727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:29.626727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:29.629527 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 481/948] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:29.636049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bc40a-9ec2-4d5c-928e-64669211474f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:29.750253 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2bc40a-9ec2-4d5c-928e-64669211474f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:29.785629 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340a043-279f-4a53-bbb4-51b35285b0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:29.786301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340a043-279f-4a53-bbb4-51b35285b0df None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:29.786577 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340a043-279f-4a53-bbb4-51b35285b0df None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:29.786844 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340a043-279f-4a53-bbb4-51b35285b0df None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:29.787599 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340a043-279f-4a53-bbb4-51b35285b0df None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:29.787962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340a043-279f-4a53-bbb4-51b35285b0df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:29.789497 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340a043-279f-4a53-bbb4-51b35285b0df None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:29.800445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:29.808073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 468/949] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Jun 25 15:54:29 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:30.704556 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368de4e-e1d4-436d-a9eb-d3e69e97593c None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:30.705086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368de4e-e1d4-436d-a9eb-d3e69e97593c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:30.705086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368de4e-e1d4-436d-a9eb-d3e69e97593c None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:30.709484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 482/950] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:30.714797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf0876-ac62-4f1b-8e71-ff25212bf37e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:30.715203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf0876-ac62-4f1b-8e71-ff25212bf37e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:30.715359 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf0876-ac62-4f1b-8e71-ff25212bf37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:30.733289 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-cdbf0876-ac62-4f1b-8e71-ff25212bf37e None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:30.734472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-cdbf0876-ac62-4f1b-8e71-ff25212bf37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:30.807282 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 469/951] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:30.830229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368de4e-e1d4-436d-a9eb-d3e69e97593c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:30.939485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368de4e-e1d4-436d-a9eb-d3e69e97593c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:30.980894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822fd81d-cf11-4dcc-9edd-66e9b5059b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:30.981732 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fd81d-cf11-4dcc-9edd-66e9b5059b28 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:30.982024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fd81d-cf11-4dcc-9edd-66e9b5059b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:30.982295 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822fd81d-cf11-4dcc-9edd-66e9b5059b28 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:30.982957 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822fd81d-cf11-4dcc-9edd-66e9b5059b28 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:30.984321 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822fd81d-cf11-4dcc-9edd-66e9b5059b28 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:30.997091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-822fd81d-cf11-4dcc-9edd-66e9b5059b28 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 25 15:54:30.997091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:30.997091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:30.997091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:30.997091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:30.997091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:30.997091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:30.997091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:30.997091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 25 15:54:30.997091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:30.997091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:30.997091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:30.997091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:30.997091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:30.997091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:30.997091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:30.997091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:30.998095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:30.998095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:30.998095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:30.998095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:30.998095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:30.998095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:30.998095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:30.998095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:30.998095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:30.998095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:30.998095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:30.998095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:30.998095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:30.998095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:30.998095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:30.998095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:30.998949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:30.998949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:30.998949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:30.998949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:30.998949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:30.998949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:30.998949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:30.998949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:30.998949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:30.998949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:30.998949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:30.998949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:30.998949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 25 15:54:30.998949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:30.998949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 483/952] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Jun 25 15:54:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:31.004592 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf0876-ac62-4f1b-8e71-ff25212bf37e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:31.106168 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf0876-ac62-4f1b-8e71-ff25212bf37e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:31.124618 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb8a40-07c8-4e12-94d6-22b4d08088ca None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:31.124952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb8a40-07c8-4e12-94d6-22b4d08088ca None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:31.125169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb8a40-07c8-4e12-94d6-22b4d08088ca None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:31.126894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 484/953] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:54:31.131854 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac20e4-4738-4578-9c6e-846d306b6952 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:31.132145 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac20e4-4738-4578-9c6e-846d306b6952 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:31.132434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac20e4-4738-4578-9c6e-846d306b6952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:31.136097 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:31.141195 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:31.145099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6a5d0-5a5d-47f1-8597-bcb5f484bd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:31.146196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6a5d0-5a5d-47f1-8597-bcb5f484bd78 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:31.146871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6a5d0-5a5d-47f1-8597-bcb5f484bd78 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:31.147143 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6a5d0-5a5d-47f1-8597-bcb5f484bd78 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:31.147860 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c6a5d0-5a5d-47f1-8597-bcb5f484bd78 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:31.148191 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c6a5d0-5a5d-47f1-8597-bcb5f484bd78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:31.149616 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c6a5d0-5a5d-47f1-8597-bcb5f484bd78 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:31.156454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 470/954] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Jun 25 15:54:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:31.160473 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-aeac20e4-4738-4578-9c6e-846d306b6952 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:31.160592 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6a5d0-5a5d-47f1-8597-bcb5f484bd78 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:31.162333 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-aeac20e4-4738-4578-9c6e-846d306b6952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:31.268605 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 485/955] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:31.293083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac20e4-4738-4578-9c6e-846d306b6952 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:31.297270 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6a5d0-5a5d-47f1-8597-bcb5f484bd78 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:31.333248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e7a4d-6cf5-475a-9af7-3708fcd7a296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:31.334350 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e7a4d-6cf5-475a-9af7-3708fcd7a296 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:31.334870 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e7a4d-6cf5-475a-9af7-3708fcd7a296 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:31.335341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e7a4d-6cf5-475a-9af7-3708fcd7a296 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:31.336476 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e7a4d-6cf5-475a-9af7-3708fcd7a296 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:31.336992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e7a4d-6cf5-475a-9af7-3708fcd7a296 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:31.338594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e7a4d-6cf5-475a-9af7-3708fcd7a296 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:31.344367 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 471/956] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Jun 25 15:54:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:31.350304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e7a4d-6cf5-475a-9af7-3708fcd7a296 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:31.410337 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:31.414026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac20e4-4738-4578-9c6e-846d306b6952 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:31.451569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e007bb-3cdb-49f5-bc97-37649ac4ed3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:31.452372 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e007bb-3cdb-49f5-bc97-37649ac4ed3b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:31.452673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e007bb-3cdb-49f5-bc97-37649ac4ed3b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:31.452943 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e007bb-3cdb-49f5-bc97-37649ac4ed3b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:31.453695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e007bb-3cdb-49f5-bc97-37649ac4ed3b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:31.455238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e007bb-3cdb-49f5-bc97-37649ac4ed3b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:31.457267 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:31.461435 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e7a4d-6cf5-475a-9af7-3708fcd7a296 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:31.470251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-74e007bb-3cdb-49f5-bc97-37649ac4ed3b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 25 15:54:31.470251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:31.470251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:31.470251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:31.470251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:31.470251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:31.470251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:31.470251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:31.470251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 25 15:54:31.470251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:31.470251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:31.470251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:31.470251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:31.470251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:31.470251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:31.470251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:31.470251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:31.471382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:31.471382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:31.471382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:31.471382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:31.471382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:31.471382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:31.471382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:31.471382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:31.471382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:31.471382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:31.471382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:31.471382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:31.471382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:31.471382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:31.471382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:31.471382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:31.472286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:31.472286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:31.472286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:31.472286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:31.472286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:31.472286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:31.472286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:31.472286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:31.472286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:31.472286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:31.472286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:31.472286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:31.472286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 25 15:54:31.472286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:31.477077 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 486/957] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:31.478194 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e007bb-3cdb-49f5-bc97-37649ac4ed3b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:31.510188 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f38ac-b62d-4a86-95a7-69e60cc3c617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:31.510863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f38ac-b62d-4a86-95a7-69e60cc3c617 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:31.511184 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f38ac-b62d-4a86-95a7-69e60cc3c617 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:31.511712 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f38ac-b62d-4a86-95a7-69e60cc3c617 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:31.512989 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f38ac-b62d-4a86-95a7-69e60cc3c617 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:31.514658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f38ac-b62d-4a86-95a7-69e60cc3c617 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:31.548523 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 472/958] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Jun 25 15:54:31 2022] POST /identity/v3/endpoints => generated 349 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:54:31.614001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a0bda-11f4-49b1-9a7a-e604557b9ff0 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:31.614560 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a0bda-11f4-49b1-9a7a-e604557b9ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:31.614840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a0bda-11f4-49b1-9a7a-e604557b9ff0 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:31.616402 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e007bb-3cdb-49f5-bc97-37649ac4ed3b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:31.616583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 473/959] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:31.624152 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7017b-5423-4c48-916d-8204f42d2099 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:31.624609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7017b-5423-4c48-916d-8204f42d2099 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:31.624705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7017b-5423-4c48-916d-8204f42d2099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:31.645809 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-43b7017b-5423-4c48-916d-8204f42d2099 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:31.647389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-43b7017b-5423-4c48-916d-8204f42d2099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:31.676317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fc51a-4f08-4c2a-aa3a-713bdf6bf6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:31.677033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fc51a-4f08-4c2a-aa3a-713bdf6bf6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:31.677312 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fc51a-4f08-4c2a-aa3a-713bdf6bf6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:31.677585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fc51a-4f08-4c2a-aa3a-713bdf6bf6e8 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:31.678577 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fc51a-4f08-4c2a-aa3a-713bdf6bf6e8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:31.678884 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fc51a-4f08-4c2a-aa3a-713bdf6bf6e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:31.681179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fc51a-4f08-4c2a-aa3a-713bdf6bf6e8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:31.689045 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 487/960] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Jun 25 15:54:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:31.694816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fc51a-4f08-4c2a-aa3a-713bdf6bf6e8 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:31.731788 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 474/961] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:31.757240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7017b-5423-4c48-916d-8204f42d2099 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:31.775130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:31.807632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:31.838678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fc51a-4f08-4c2a-aa3a-713bdf6bf6e8 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:31.874760 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7017b-5423-4c48-916d-8204f42d2099 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:31.875730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba4655-463a-48f4-93ef-0b44dc06784b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:31.876465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba4655-463a-48f4-93ef-0b44dc06784b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:31.876710 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba4655-463a-48f4-93ef-0b44dc06784b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:31.876989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba4655-463a-48f4-93ef-0b44dc06784b None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:31.877708 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba4655-463a-48f4-93ef-0b44dc06784b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:31.878017 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba4655-463a-48f4-93ef-0b44dc06784b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:31.879456 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba4655-463a-48f4-93ef-0b44dc06784b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:31.884778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 488/962] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Jun 25 15:54:31 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:31.888462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cfedd-6f49-4ae8-8c84-6d7666df22cd None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:31.888462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cfedd-6f49-4ae8-8c84-6d7666df22cd None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:31.888462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cfedd-6f49-4ae8-8c84-6d7666df22cd None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:31.888462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 489/963] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:31.898366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fc21f-68c0-444f-8a82-5992fc0a4fd4 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:31.901330 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fc21f-68c0-444f-8a82-5992fc0a4fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:31.901752 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fc21f-68c0-444f-8a82-5992fc0a4fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:31.922693 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece6702-2259-4782-8d9d-a1bab11a6876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:31.923985 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece6702-2259-4782-8d9d-a1bab11a6876 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:31.924539 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece6702-2259-4782-8d9d-a1bab11a6876 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:31.924664 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-2c2fc21f-68c0-444f-8a82-5992fc0a4fd4 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:31.925114 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece6702-2259-4782-8d9d-a1bab11a6876 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:31.925816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2c2fc21f-68c0-444f-8a82-5992fc0a4fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:31.938788 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece6702-2259-4782-8d9d-a1bab11a6876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:54:31.948589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-8ece6702-2259-4782-8d9d-a1bab11a6876 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 25 15:54:31.948589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:31.948589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:31.948589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:31.948589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:31.948589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:31.948589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:31.948589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:31.948589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:31.948589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:31.948589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:31.948589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:31.948589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:54:31.948589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:54:31.948589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:54:31.948589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:54:31.949625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:54:31.949625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:54:31.949625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:54:31.949625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:54:31.949625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:31.949625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:31.949625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:31.949625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:31.949625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:31.949625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:31.949625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:31.949625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:31.949625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:31.949625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:31.949625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:31.949625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:31.950533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:31.950533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:31.950533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:31.950533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:31.950533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:54:31.950533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:54:31.950533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:54:31.950533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:54:31.950533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:54:31.950533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:54:31.950533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 25 15:54:31.950533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:31.956573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 475/964] 10.222.0.13 () {56 vars in 1120 bytes} [Sat Jun 25 15:54:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:31.959927 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece6702-2259-4782-8d9d-a1bab11a6876 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:32.024573 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 490/965] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:32.053726 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fc21f-68c0-444f-8a82-5992fc0a4fd4 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:32.104311 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece6702-2259-4782-8d9d-a1bab11a6876 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:32.115639 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:32.140794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:32.152854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5f325-777f-45e0-af41-601bf61a8e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:32.153528 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5f325-777f-45e0-af41-601bf61a8e7f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:32.154311 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5f325-777f-45e0-af41-601bf61a8e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:32.154605 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5f325-777f-45e0-af41-601bf61a8e7f None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:32.155356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5f325-777f-45e0-af41-601bf61a8e7f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:32.155668 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5f325-777f-45e0-af41-601bf61a8e7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:32.157400 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5f325-777f-45e0-af41-601bf61a8e7f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:32.167274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fc21f-68c0-444f-8a82-5992fc0a4fd4 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:32.169840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 476/966] 10.222.0.13 () {56 vars in 1115 bytes} [Sat Jun 25 15:54:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:32.177341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5f325-777f-45e0-af41-601bf61a8e7f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:32.214414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065ef65-350b-4ed8-b2d5-e9da6da2ff81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:32.215661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065ef65-350b-4ed8-b2d5-e9da6da2ff81 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:32.216048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065ef65-350b-4ed8-b2d5-e9da6da2ff81 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:32.216714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065ef65-350b-4ed8-b2d5-e9da6da2ff81 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:32.217533 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065ef65-350b-4ed8-b2d5-e9da6da2ff81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:32.220004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065ef65-350b-4ed8-b2d5-e9da6da2ff81 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:32.237395 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.sql.core [None req-1065ef65-350b-4ed8-b2d5-e9da6da2ff81 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 25 15:54:32.237395 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 25 15:54:32.237395 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [parameters: {'user_id': 'd77dfedb569b43268e69781717794f0b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 25 15:54:32.237395 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80124) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 25 15:54:32.248048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-1065ef65-350b-4ed8-b2d5-e9da6da2ff81 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 25 15:54:32.248048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:32.248048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 25 15:54:32.248048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 25 15:54:32.248048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 25 15:54:32.248048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 25 15:54:32.248048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 25 15:54:32.248048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application result = self._query(query) Jun 25 15:54:32.248048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 25 15:54:32.248048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application conn.query(q) Jun 25 15:54:32.248048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 25 15:54:32.248048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 25 15:54:32.248048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 25 15:54:32.248048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application result.read() Jun 25 15:54:32.248048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 25 15:54:32.249172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 25 15:54:32.249172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 25 15:54:32.249172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 25 15:54:32.249172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 25 15:54:32.249172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 25 15:54:32.249172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 25 15:54:32.249172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 25 15:54:32.249172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 25 15:54:32.249172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:32.249172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 25 15:54:32.249172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:32.249172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:32.249172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 25 15:54:32.249172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 25 15:54:32.249172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 25 15:54:32.249172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 25 15:54:32.249172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 25 15:54:32.249172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application next(self.gen) Jun 25 15:54:32.250351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 25 15:54:32.250351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application yield resource Jun 25 15:54:32.250351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 25 15:54:32.250351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application next(self.gen) Jun 25 15:54:32.250351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 25 15:54:32.250351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self.session.rollback() Jun 25 15:54:32.250351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 25 15:54:32.250351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self.force_reraise() Jun 25 15:54:32.250351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 25 15:54:32.250351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise self.value Jun 25 15:54:32.250351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 25 15:54:32.250351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 25 15:54:32.250351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 25 15:54:32.250351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application session.commit() Jun 25 15:54:32.250351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 25 15:54:32.250351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 25 15:54:32.250351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 25 15:54:32.251525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._prepare_impl() Jun 25 15:54:32.251525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 25 15:54:32.251525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self.session.flush() Jun 25 15:54:32.251525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 25 15:54:32.251525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._flush(objects) Jun 25 15:54:32.251525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 25 15:54:32.251525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 25 15:54:32.251525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 25 15:54:32.251525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application compat.raise_( Jun 25 15:54:32.251525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 25 15:54:32.251525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception Jun 25 15:54:32.251525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 25 15:54:32.251525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application flush_context.execute() Jun 25 15:54:32.251525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 25 15:54:32.251525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rec.execute(self) Jun 25 15:54:32.251525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 25 15:54:32.251525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 25 15:54:32.252770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 25 15:54:32.252770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 25 15:54:32.252770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 25 15:54:32.252770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 25 15:54:32.252770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 25 15:54:32.252770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 25 15:54:32.252770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 25 15:54:32.252770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 25 15:54:32.252770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 25 15:54:32.252770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 25 15:54:32.252770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 25 15:54:32.252770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 25 15:54:32.252770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 25 15:54:32.252770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 25 15:54:32.252770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 25 15:54:32.252770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception Jun 25 15:54:32.253816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 25 15:54:32.253816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 25 15:54:32.253816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 25 15:54:32.253816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 25 15:54:32.253816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 25 15:54:32.253816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application result = self._query(query) Jun 25 15:54:32.253816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 25 15:54:32.253816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application conn.query(q) Jun 25 15:54:32.253816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 25 15:54:32.253816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 25 15:54:32.253816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 25 15:54:32.253816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application result.read() Jun 25 15:54:32.253816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 25 15:54:32.253816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 25 15:54:32.253816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 25 15:54:32.253816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 25 15:54:32.254855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 25 15:54:32.254855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 25 15:54:32.254855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 25 15:54:32.254855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 25 15:54:32.254855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 25 15:54:32.254855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 25 15:54:32.254855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd77dfedb569b43268e69781717794f0b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 25 15:54:32.254855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 25 15:54:32.254855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:32.254855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:32.254855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:32.254855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:32.254855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:32.254855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:32.254855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:32.255962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:32.255962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:32.255962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:32.255962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:32.255962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:32.255962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:32.255962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:32.255962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 25 15:54:32.255962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 25 15:54:32.255962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:32.255962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:32.255962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:54:32.255962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:32.255962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:54:32.255962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:32.255962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 25 15:54:32.255962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 25 15:54:32.257227 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 25 15:54:32.257227 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 25 15:54:32.257227 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 25 15:54:32.257227 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 25 15:54:32.257227 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 25 15:54:32.257227 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:32.258441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 491/967] 10.222.0.13 () {60 vars in 1131 bytes} [Sat Jun 25 15:54:32 2022] POST /identity/v3/users => generated 160 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:32.264327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065ef65-350b-4ed8-b2d5-e9da6da2ff81 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:32.314395 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5f325-777f-45e0-af41-601bf61a8e7f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:32.355869 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c0c3f-4922-4c46-80a0-46913a969cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:32.356614 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c0c3f-4922-4c46-80a0-46913a969cfc None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:32.356902 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c0c3f-4922-4c46-80a0-46913a969cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:32.357177 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c0c3f-4922-4c46-80a0-46913a969cfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:32.369398 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c0c3f-4922-4c46-80a0-46913a969cfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:32.370803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c0c3f-4922-4c46-80a0-46913a969cfc None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:32.390976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-dd9c0c3f-4922-4c46-80a0-46913a969cfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:32.390976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:32.390976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:32.390976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:32.390976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:32.390976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:32.390976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:32.390976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:32.390976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:32.390976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:32.390976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:32.390976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:32.390976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:32.390976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:32.390976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:32.392375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:32.392375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:32.392375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:32.392375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:32.392375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:32.392375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:32.392375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:32.392375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:32.392375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:32.392375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:32.392375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:32.392375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:32.392375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:32.392375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:32.392375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:32.392375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:32.392375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:32.393391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:32.393391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:32.393391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:32.393391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:32.393391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 477/968] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:32.395890 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065ef65-350b-4ed8-b2d5-e9da6da2ff81 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:32.398891 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c0c3f-4922-4c46-80a0-46913a969cfc None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:32.416734 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:32.418954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:32.441877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1cfe0-a136-4e3c-a157-9d0f2ecd942b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:32.443010 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1cfe0-a136-4e3c-a157-9d0f2ecd942b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:32.443432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1cfe0-a136-4e3c-a157-9d0f2ecd942b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:32.443842 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1cfe0-a136-4e3c-a157-9d0f2ecd942b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:32.452172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1cfe0-a136-4e3c-a157-9d0f2ecd942b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:32.454020 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1cfe0-a136-4e3c-a157-9d0f2ecd942b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:32.468488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-e6d1cfe0-a136-4e3c-a157-9d0f2ecd942b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 25 15:54:32.468488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:32.468488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:32.468488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:32.468488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:32.468488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:32.468488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:32.468488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:32.468488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:32.468488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:32.468488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:32.468488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:32.468488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:54:32.468488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:54:32.468488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:54:32.468488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:54:32.469654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:32.469654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:32.469654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:54:32.469654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:32.469654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:54:32.469654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:32.469654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:32.469654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:32.469654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:32.469654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:32.469654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:32.469654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:32.469654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:32.469654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:32.469654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:32.469654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:32.469654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:32.469654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:32.470735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:32.470735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:32.470735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:54:32.470735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:54:32.470735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:54:32.470735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:54:32.470735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:54:32.470735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:54:32.470735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 25 15:54:32.470735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:32.473046 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 492/969] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:54:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:32.481841 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1cfe0-a136-4e3c-a157-9d0f2ecd942b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:32.505268 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c0c3f-4922-4c46-80a0-46913a969cfc None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:32.540648 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9d92b-1d3c-4b3a-ae97-a9c2f9579650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:32.541452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9d92b-1d3c-4b3a-ae97-a9c2f9579650 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:32.541803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9d92b-1d3c-4b3a-ae97-a9c2f9579650 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:32.542175 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9d92b-1d3c-4b3a-ae97-a9c2f9579650 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:32.542986 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9d92b-1d3c-4b3a-ae97-a9c2f9579650 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:32.543361 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9d92b-1d3c-4b3a-ae97-a9c2f9579650 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:32.545017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9d92b-1d3c-4b3a-ae97-a9c2f9579650 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:32.562072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 478/970] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:32.563457 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9d92b-1d3c-4b3a-ae97-a9c2f9579650 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:32.606938 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1cfe0-a136-4e3c-a157-9d0f2ecd942b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:32.642209 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4b6ad-878f-461e-8083-e2672c962506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:32.643059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4b6ad-878f-461e-8083-e2672c962506 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:32.643432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4b6ad-878f-461e-8083-e2672c962506 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:32.643801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4b6ad-878f-461e-8083-e2672c962506 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:32.644012 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:32.644769 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4b6ad-878f-461e-8083-e2672c962506 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:32.645137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4b6ad-878f-461e-8083-e2672c962506 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:32.646621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4b6ad-878f-461e-8083-e2672c962506 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:32.668970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9d92b-1d3c-4b3a-ae97-a9c2f9579650 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:32.686465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 493/971] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Jun 25 15:54:32 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:32.706225 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1d65f-405a-4e09-a45a-4fb72ed32b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:32.707069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1d65f-405a-4e09-a45a-4fb72ed32b08 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:32.707353 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1d65f-405a-4e09-a45a-4fb72ed32b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:32.707628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf1d65f-405a-4e09-a45a-4fb72ed32b08 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:32.716689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1d65f-405a-4e09-a45a-4fb72ed32b08 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:32.718183 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf1d65f-405a-4e09-a45a-4fb72ed32b08 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:32.735005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-3bf1d65f-405a-4e09-a45a-4fb72ed32b08 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 25 15:54:32.735005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:32.735005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:32.735005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:32.735005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:32.735005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:32.735005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:32.735005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:32.735005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:32.735005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:32.735005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:32.735005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:32.735005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:54:32.735005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:54:32.735005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:54:32.735005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:54:32.736450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:32.736450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:32.736450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:54:32.736450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:32.736450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:54:32.736450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:32.736450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:32.736450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:32.736450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:32.736450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:32.736450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:32.736450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:32.736450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:32.736450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:32.736450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:32.736450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:32.736450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:32.736450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:32.737413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:32.737413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:32.737413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:54:32.737413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:54:32.737413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:54:32.737413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:54:32.737413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:54:32.737413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:54:32.737413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 25 15:54:32.737413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:32.740713 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 479/972] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Jun 25 15:54:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:32.742767 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4b6ad-878f-461e-8083-e2672c962506 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:32.780729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:32.856337 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4b6ad-878f-461e-8083-e2672c962506 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:32.896901 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c17cee-e4fb-43ad-9f95-990385e2cfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:32.897674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c17cee-e4fb-43ad-9f95-990385e2cfe6 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:32.897964 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c17cee-e4fb-43ad-9f95-990385e2cfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:32.898397 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c17cee-e4fb-43ad-9f95-990385e2cfe6 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:32.899239 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c17cee-e4fb-43ad-9f95-990385e2cfe6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:32.899555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c17cee-e4fb-43ad-9f95-990385e2cfe6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:32.901322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c17cee-e4fb-43ad-9f95-990385e2cfe6 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:32.924500 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 494/973] 10.222.0.13 () {56 vars in 1149 bytes} [Sat Jun 25 15:54:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:32.926121 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1d65f-405a-4e09-a45a-4fb72ed32b08 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:33.053471 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf1d65f-405a-4e09-a45a-4fb72ed32b08 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:33.092645 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fab20-2431-4f9a-99a0-ab2b3d72cad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:33.093654 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fab20-2431-4f9a-99a0-ab2b3d72cad5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:33.094135 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fab20-2431-4f9a-99a0-ab2b3d72cad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:33.094624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5fab20-2431-4f9a-99a0-ab2b3d72cad5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:33.098496 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:33.169065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fab20-2431-4f9a-99a0-ab2b3d72cad5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:33.170453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5fab20-2431-4f9a-99a0-ab2b3d72cad5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:33.281928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 480/974] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:54:32 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:54:33.283237 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c17cee-e4fb-43ad-9f95-990385e2cfe6 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:33.378199 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:33.390456 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c17cee-e4fb-43ad-9f95-990385e2cfe6 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:33.428300 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d232e2-e32e-4562-bea4-f8bf59536056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:33.429046 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d232e2-e32e-4562-bea4-f8bf59536056 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:33.429371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d232e2-e32e-4562-bea4-f8bf59536056 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:33.429855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d232e2-e32e-4562-bea4-f8bf59536056 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:33.440437 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d232e2-e32e-4562-bea4-f8bf59536056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:33.441907 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d232e2-e32e-4562-bea4-f8bf59536056 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:33.459695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-49d232e2-e32e-4562-bea4-f8bf59536056 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:33.459695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:33.459695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:33.459695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:33.459695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:33.459695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:33.459695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:33.459695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:33.459695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:33.459695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:33.459695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:33.459695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:33.459695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:33.459695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:33.459695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:33.468273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:33.468273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:33.468273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:33.468273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:33.468273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:33.468273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:33.468273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:33.468273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:33.468273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:33.468273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:33.468273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:33.468273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:33.468273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:33.468273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:33.468273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:33.468273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:33.468273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:33.469215 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:33.469215 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:33.469215 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:33.469215 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:33.469215 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 495/975] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:33.469506 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fab20-2431-4f9a-99a0-ab2b3d72cad5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:33.506442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:33.575499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5fab20-2431-4f9a-99a0-ab2b3d72cad5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:33.612537 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb427f-42cc-4348-8c5a-03a026443ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:33.613352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb427f-42cc-4348-8c5a-03a026443ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:33.613725 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb427f-42cc-4348-8c5a-03a026443ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:33.614094 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb427f-42cc-4348-8c5a-03a026443ac4 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:33.614918 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb427f-42cc-4348-8c5a-03a026443ac4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:33.615321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb427f-42cc-4348-8c5a-03a026443ac4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:33.618330 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb427f-42cc-4348-8c5a-03a026443ac4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:33.634663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 481/976] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:33.635963 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d232e2-e32e-4562-bea4-f8bf59536056 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:33.742918 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d232e2-e32e-4562-bea4-f8bf59536056 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:33.779974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792f51a-cbbb-41ec-9dec-aa76e7046dab None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:33.780768 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792f51a-cbbb-41ec-9dec-aa76e7046dab None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:33.780768 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792f51a-cbbb-41ec-9dec-aa76e7046dab None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:33.782184 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 482/977] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:54:33.782320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca0ed3-4a0b-47a1-a651-9bee8a4e1f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:33.783453 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca0ed3-4a0b-47a1-a651-9bee8a4e1f62 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:33.784008 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca0ed3-4a0b-47a1-a651-9bee8a4e1f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:33.784524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca0ed3-4a0b-47a1-a651-9bee8a4e1f62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:33.787467 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f7ec6-cea6-4334-981c-ffa05d1020b4 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:33.787753 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f7ec6-cea6-4334-981c-ffa05d1020b4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:33.788056 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f7ec6-cea6-4334-981c-ffa05d1020b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:33.798138 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aca0ed3-4a0b-47a1-a651-9bee8a4e1f62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:33.799588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aca0ed3-4a0b-47a1-a651-9bee8a4e1f62 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:33.805976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-462f7ec6-cea6-4334-981c-ffa05d1020b4 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:33.807185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-462f7ec6-cea6-4334-981c-ffa05d1020b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:33.814504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-6aca0ed3-4a0b-47a1-a651-9bee8a4e1f62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:33.814504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:33.814504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:33.814504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:33.814504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:33.814504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:33.814504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:33.814504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:33.814504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:33.814504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:33.814504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:33.814504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:33.814504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:54:33.814504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:54:33.814504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:54:33.815673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:54:33.815673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:33.815673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:33.815673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:33.815673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:33.815673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:33.815673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:33.815673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:33.815673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:33.815673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:33.815673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:33.815673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:33.815673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:33.815673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:33.815673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:33.815673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:33.815673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:33.816688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:54:33.816688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:54:33.816688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:54:33.816688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:54:33.816688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:54:33.816688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:54:33.816688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:33.816688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:33.820370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 496/978] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:33.821909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca0ed3-4a0b-47a1-a651-9bee8a4e1f62 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:33.824620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:33.885029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 483/979] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:33.907893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f7ec6-cea6-4334-981c-ffa05d1020b4 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:33.933611 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca0ed3-4a0b-47a1-a651-9bee8a4e1f62 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:33.940121 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:33.967094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6c536-fb9c-401a-94b0-134f339bcacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:33.967756 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6c536-fb9c-401a-94b0-134f339bcacc None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:33.968033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6c536-fb9c-401a-94b0-134f339bcacc None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:33.968310 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6c536-fb9c-401a-94b0-134f339bcacc None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:33.969134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd6c536-fb9c-401a-94b0-134f339bcacc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:33.969439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd6c536-fb9c-401a-94b0-134f339bcacc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:33.970882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd6c536-fb9c-401a-94b0-134f339bcacc None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:33.984115 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 497/980] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:54:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:33.990120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6c536-fb9c-401a-94b0-134f339bcacc None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:34.028632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f7ec6-cea6-4334-981c-ffa05d1020b4 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:34.053418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:34.063460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb0a13-535c-4eba-bfa7-41ff963673ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:34.064269 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb0a13-535c-4eba-bfa7-41ff963673ad None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:34.064584 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb0a13-535c-4eba-bfa7-41ff963673ad None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:34.064860 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb0a13-535c-4eba-bfa7-41ff963673ad None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:34.065524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb0a13-535c-4eba-bfa7-41ff963673ad None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:34.066917 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb0a13-535c-4eba-bfa7-41ff963673ad None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:34.079396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-abcb0a13-535c-4eba-bfa7-41ff963673ad None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 25 15:54:34.079396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:34.079396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:34.079396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:34.079396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:34.079396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:34.079396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:34.079396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:34.079396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 25 15:54:34.079396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:34.079396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:34.079396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:34.079396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:34.079396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:34.079396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:34.079396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:34.079396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:34.080595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:34.080595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:34.080595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:34.080595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:34.080595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:34.080595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:34.080595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:34.080595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:34.080595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:34.080595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:34.080595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:34.080595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:34.080595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:34.080595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:34.080595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:34.080595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:34.081425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:34.081425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:34.081425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:34.081425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:34.081425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:34.081425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:34.081425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:34.081425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:34.081425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:34.081425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:34.081425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:34.081425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:34.081425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 25 15:54:34.081425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:34.081425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 484/981] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Jun 25 15:54:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:34.082255 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8960e1-ffb3-4eac-80d5-3ff121f94199 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:34.082544 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8960e1-ffb3-4eac-80d5-3ff121f94199 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:34.082820 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8960e1-ffb3-4eac-80d5-3ff121f94199 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:34.084213 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 485/982] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:34.086580 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8960e1-ffb3-4eac-80d5-3ff121f94199 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:34.099380 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6c536-fb9c-401a-94b0-134f339bcacc None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:34.134509 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4b7ef-02cb-42bd-9a87-273eb8953319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:34.135160 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4b7ef-02cb-42bd-9a87-273eb8953319 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:34.135441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4b7ef-02cb-42bd-9a87-273eb8953319 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:34.135715 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4b7ef-02cb-42bd-9a87-273eb8953319 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:34.136579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4b7ef-02cb-42bd-9a87-273eb8953319 None admin] RBAC: Adding query filter params (role.id=c38990a9778e4b2abd3376f9e066d3a5, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=28b3e73439e64050b406a9f3e9b0160a) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:34.136894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4b7ef-02cb-42bd-9a87-273eb8953319 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:34.138301 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca4b7ef-02cb-42bd-9a87-273eb8953319 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:34.144737 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 498/983] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:54:33 2022] GET /identity/v3/role_assignments?user.id=28b3e73439e64050b406a9f3e9b0160a&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=c38990a9778e4b2abd3376f9e066d3a5 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:34.146013 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba8a68-640a-4fcc-aba1-da18cadaebac None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:34.146299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba8a68-640a-4fcc-aba1-da18cadaebac None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:34.146573 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba8a68-640a-4fcc-aba1-da18cadaebac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:34.164709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8aba8a68-640a-4fcc-aba1-da18cadaebac None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:34.166392 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8aba8a68-640a-4fcc-aba1-da18cadaebac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:34.186394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:34.200215 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8960e1-ffb3-4eac-80d5-3ff121f94199 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:34.236512 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfeec79-6772-4914-a4b7-1d2adcad3362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:34.237186 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfeec79-6772-4914-a4b7-1d2adcad3362 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:34.237466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfeec79-6772-4914-a4b7-1d2adcad3362 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:34.237736 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfeec79-6772-4914-a4b7-1d2adcad3362 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:34.238453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfeec79-6772-4914-a4b7-1d2adcad3362 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:34.238756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfeec79-6772-4914-a4b7-1d2adcad3362 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:34.240187 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfeec79-6772-4914-a4b7-1d2adcad3362 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:34.243881 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 499/984] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:34.250023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 486/985] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Jun 25 15:54:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:34.253869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba8a68-640a-4fcc-aba1-da18cadaebac None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:34.269289 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfeec79-6772-4914-a4b7-1d2adcad3362 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:34.321552 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:34.373916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba8a68-640a-4fcc-aba1-da18cadaebac None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:34.384995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfeec79-6772-4914-a4b7-1d2adcad3362 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:34.411005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e4f6d-1a06-4f2c-8ea8-9519b5fdd459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:34.411672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e4f6d-1a06-4f2c-8ea8-9519b5fdd459 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:34.411957 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e4f6d-1a06-4f2c-8ea8-9519b5fdd459 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:34.412235 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e4f6d-1a06-4f2c-8ea8-9519b5fdd459 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:34.412985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e4f6d-1a06-4f2c-8ea8-9519b5fdd459 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:34.413299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e4f6d-1a06-4f2c-8ea8-9519b5fdd459 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:34.414851 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e4f6d-1a06-4f2c-8ea8-9519b5fdd459 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:34.419901 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 500/986] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Jun 25 15:54:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:34.423399 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73880b36-c82c-40ad-ba43-30eea4808cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:34.424170 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73880b36-c82c-40ad-ba43-30eea4808cb4 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:34.424753 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73880b36-c82c-40ad-ba43-30eea4808cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:34.424932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73880b36-c82c-40ad-ba43-30eea4808cb4 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:34.425608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e4f6d-1a06-4f2c-8ea8-9519b5fdd459 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:34.425856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73880b36-c82c-40ad-ba43-30eea4808cb4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:34.427321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73880b36-c82c-40ad-ba43-30eea4808cb4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:34.443361 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 487/987] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Jun 25 15:54:34 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 15:54:34.545576 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e4f6d-1a06-4f2c-8ea8-9519b5fdd459 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:34.567886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:34.591769 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c13ff-bdfa-406b-a591-8423d4157257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:34.592461 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c13ff-bdfa-406b-a591-8423d4157257 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:34.592821 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c13ff-bdfa-406b-a591-8423d4157257 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:34.593167 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c13ff-bdfa-406b-a591-8423d4157257 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:34.594045 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c13ff-bdfa-406b-a591-8423d4157257 None admin] RBAC: Adding query filter params (interface=admin, service_id=74b93a1263f442ec9fb104dcd03c0463, region_id=RegionOne) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:34.594430 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c13ff-bdfa-406b-a591-8423d4157257 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:34.595872 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c13ff-bdfa-406b-a591-8423d4157257 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:34.601894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 501/988] 10.222.0.13 () {56 vars in 1249 bytes} [Sat Jun 25 15:54:34 2022] GET /identity/v3/endpoints?service_id=74b93a1263f442ec9fb104dcd03c0463&interface=admin®ion_id=RegionOne => generated 193 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:34.607459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73880b36-c82c-40ad-ba43-30eea4808cb4 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:34.625017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:34.713127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73880b36-c82c-40ad-ba43-30eea4808cb4 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:34.748505 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e883a17-3d36-436c-a26c-34118233d47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:34.749180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e883a17-3d36-436c-a26c-34118233d47f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:34.749461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e883a17-3d36-436c-a26c-34118233d47f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:34.749734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e883a17-3d36-436c-a26c-34118233d47f None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:34.750422 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e883a17-3d36-436c-a26c-34118233d47f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:34.751793 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e883a17-3d36-436c-a26c-34118233d47f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:34.758276 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 488/989] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Jun 25 15:54:34 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 15:54:36.190451 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5bad8-0022-48a4-b97a-64319b25aeef None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:36.190991 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5bad8-0022-48a4-b97a-64319b25aeef None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:36.190991 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5bad8-0022-48a4-b97a-64319b25aeef None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:36.192351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 502/990] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:36.197989 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263a551-6a70-45a6-98d4-b1ac3c607849 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:36.198405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263a551-6a70-45a6-98d4-b1ac3c607849 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:36.198544 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263a551-6a70-45a6-98d4-b1ac3c607849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:36.216396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-3263a551-6a70-45a6-98d4-b1ac3c607849 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:36.217624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3263a551-6a70-45a6-98d4-b1ac3c607849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:36.288840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 489/991] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:36.311496 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5bad8-0022-48a4-b97a-64319b25aeef None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:36.420165 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5bad8-0022-48a4-b97a-64319b25aeef None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:36.455833 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca80020-08a7-42f2-a614-531897e3d8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:36.456572 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca80020-08a7-42f2-a614-531897e3d8de None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:36.456863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca80020-08a7-42f2-a614-531897e3d8de None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:36.457114 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca80020-08a7-42f2-a614-531897e3d8de None admin] PATH_INFO: `/v3/services/image` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:36.457766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca80020-08a7-42f2-a614-531897e3d8de None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:36.459136 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca80020-08a7-42f2-a614-531897e3d8de None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:36.471152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-dca80020-08a7-42f2-a614-531897e3d8de None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 25 15:54:36.471152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:36.471152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:36.471152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:36.471152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:36.471152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:36.471152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:36.471152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:36.471152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 25 15:54:36.471152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:36.471152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:36.471152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:36.471152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:36.471152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:36.471152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:36.471152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:36.471152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:36.472249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:36.472249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:36.472249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:36.472249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:36.472249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:36.472249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:36.472249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:36.472249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:36.472249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:36.472249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:36.472249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:36.472249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:36.472249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:36.472249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:36.472249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:36.472249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:36.473175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:36.473175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:36.473175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:36.473175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:36.473175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:36.473175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:36.473175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:36.473175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:36.473175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:36.473175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:36.473175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:36.473175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:36.473175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 25 15:54:36.473175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:36.473175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 503/992] 10.222.0.13 () {56 vars in 1105 bytes} [Sat Jun 25 15:54:36 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:36.478200 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb4a6a-ee04-4560-bb25-1722bc6cc440 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:36.478476 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb4a6a-ee04-4560-bb25-1722bc6cc440 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:36.478979 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb4a6a-ee04-4560-bb25-1722bc6cc440 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:36.480147 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca80020-08a7-42f2-a614-531897e3d8de None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:36.484122 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 490/993] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:54:36.485247 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb689715-4409-4965-b440-0396991d0b01 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:36.485528 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb689715-4409-4965-b440-0396991d0b01 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:36.485795 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb689715-4409-4965-b440-0396991d0b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:36.503185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-bb689715-4409-4965-b440-0396991d0b01 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:36.504749 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bb689715-4409-4965-b440-0396991d0b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:36.507016 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:36.534930 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:36.579794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 491/994] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:36.592439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca80020-08a7-42f2-a614-531897e3d8de None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:36.604055 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb689715-4409-4965-b440-0396991d0b01 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:36.627707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561a2ca-96a5-4eea-9413-a3479232728b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:36.628373 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561a2ca-96a5-4eea-9413-a3479232728b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:36.628688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561a2ca-96a5-4eea-9413-a3479232728b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:36.628960 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561a2ca-96a5-4eea-9413-a3479232728b None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:36.629672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561a2ca-96a5-4eea-9413-a3479232728b None admin] RBAC: Adding query filter params (name=image) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:36.629980 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561a2ca-96a5-4eea-9413-a3479232728b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:36.631406 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561a2ca-96a5-4eea-9413-a3479232728b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:36.636927 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 504/995] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:36.642542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561a2ca-96a5-4eea-9413-a3479232728b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:36.712318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb689715-4409-4965-b440-0396991d0b01 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:36.746947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561a2ca-96a5-4eea-9413-a3479232728b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:36.748769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc0642-7fae-4482-af7c-e4e39d4f564e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:36.749583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc0642-7fae-4482-af7c-e4e39d4f564e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:36.749859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc0642-7fae-4482-af7c-e4e39d4f564e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:36.750136 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc0642-7fae-4482-af7c-e4e39d4f564e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:36.750794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc0642-7fae-4482-af7c-e4e39d4f564e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:36.752183 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc0642-7fae-4482-af7c-e4e39d4f564e None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:36.757954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:36.764547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-86bc0642-7fae-4482-af7c-e4e39d4f564e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 25 15:54:36.764547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:36.764547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:36.764547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:36.764547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:36.764547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:36.764547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:36.764547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:36.764547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 25 15:54:36.764547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:36.764547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:36.764547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:36.764547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:36.764547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:36.764547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:36.764547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:36.764547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:36.765549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:36.765549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:36.765549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:36.765549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:36.765549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:36.765549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:36.765549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:36.765549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:36.765549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:36.765549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:36.765549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:36.765549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:36.765549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:36.765549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:36.765549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:36.765549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:36.766707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:36.766707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:36.766707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:36.766707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:36.766707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:36.766707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:36.766707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:36.766707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:36.766707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:36.766707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:36.766707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:36.766707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:36.766707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 25 15:54:36.766707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:36.766707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 492/996] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:36.772269 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea6945-514c-4934-80e0-00beb9b35149 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:36.772578 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea6945-514c-4934-80e0-00beb9b35149 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:36.772857 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea6945-514c-4934-80e0-00beb9b35149 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:36.774260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 493/997] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:54:36.774905 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea6945-514c-4934-80e0-00beb9b35149 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:36.778293 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:36.786826 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e16554-6730-4b41-baf7-15debc54f760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:36.787554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e16554-6730-4b41-baf7-15debc54f760 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:36.787834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e16554-6730-4b41-baf7-15debc54f760 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:36.788107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e16554-6730-4b41-baf7-15debc54f760 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:36.789004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e16554-6730-4b41-baf7-15debc54f760 None admin] RBAC: Adding query filter params (type=image) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:36.789313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e16554-6730-4b41-baf7-15debc54f760 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:36.790694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e16554-6730-4b41-baf7-15debc54f760 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:36.795194 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 505/998] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:36 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:36.796342 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb6e8b-2aab-4a14-b11b-9e36465dce92 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:36.796661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb6e8b-2aab-4a14-b11b-9e36465dce92 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:36.796937 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb6e8b-2aab-4a14-b11b-9e36465dce92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:36.820144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-d8cb6e8b-2aab-4a14-b11b-9e36465dce92 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:36.821520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d8cb6e8b-2aab-4a14-b11b-9e36465dce92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:36.897710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea6945-514c-4934-80e0-00beb9b35149 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:36.904665 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 506/999] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:36.928192 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb6e8b-2aab-4a14-b11b-9e36465dce92 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:36.935350 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a8a5e-f87f-4c69-a4c0-38d51cd312a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:36.936034 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a8a5e-f87f-4c69-a4c0-38d51cd312a2 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:36.936310 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a8a5e-f87f-4c69-a4c0-38d51cd312a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:36.936608 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a8a5e-f87f-4c69-a4c0-38d51cd312a2 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:36.937334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a8a5e-f87f-4c69-a4c0-38d51cd312a2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:36.937699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a8a5e-f87f-4c69-a4c0-38d51cd312a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:36.939131 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a8a5e-f87f-4c69-a4c0-38d51cd312a2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:36.949372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 494/1000] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Jun 25 15:54:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:36.951488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a8a5e-f87f-4c69-a4c0-38d51cd312a2 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:37.036563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:37.040546 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb6e8b-2aab-4a14-b11b-9e36465dce92 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:37.042294 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:37.073152 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a8a5e-f87f-4c69-a4c0-38d51cd312a2 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:37.078119 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b97968-88c1-4c3d-810e-5ccdcc4820e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:37.079106 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b97968-88c1-4c3d-810e-5ccdcc4820e5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:37.079533 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b97968-88c1-4c3d-810e-5ccdcc4820e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:37.080069 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b97968-88c1-4c3d-810e-5ccdcc4820e5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:37.080776 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b97968-88c1-4c3d-810e-5ccdcc4820e5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:37.082259 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b97968-88c1-4c3d-810e-5ccdcc4820e5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:37.095052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-b9b97968-88c1-4c3d-810e-5ccdcc4820e5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 25 15:54:37.095052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:37.095052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:37.095052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:37.095052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:37.095052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:37.095052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:37.095052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:37.095052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 25 15:54:37.095052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:37.095052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:37.095052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:37.095052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:37.095052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:37.095052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:37.095052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:37.095052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:37.096107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:37.096107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:37.096107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:37.096107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:37.096107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:37.096107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:37.096107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:37.096107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:37.096107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:37.096107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:37.096107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:37.096107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:37.096107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:37.096107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:37.096107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:37.096107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:37.096983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:37.096983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:37.096983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:37.096983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:37.096983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:37.096983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:37.096983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:37.096983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:37.096983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:37.096983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:37.096983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:37.096983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:37.096983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 25 15:54:37.096983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:37.096983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 507/1001] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Jun 25 15:54:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:37.102357 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b97968-88c1-4c3d-810e-5ccdcc4820e5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:37.109949 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950921be-3e73-4847-b7a5-97476cc94b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:37.110626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950921be-3e73-4847-b7a5-97476cc94b46 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:37.110903 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950921be-3e73-4847-b7a5-97476cc94b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:37.111179 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950921be-3e73-4847-b7a5-97476cc94b46 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:37.111900 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950921be-3e73-4847-b7a5-97476cc94b46 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:37.112698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950921be-3e73-4847-b7a5-97476cc94b46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:37.114930 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950921be-3e73-4847-b7a5-97476cc94b46 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:37.121682 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 495/1002] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Jun 25 15:54:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:37.127175 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950921be-3e73-4847-b7a5-97476cc94b46 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:37.210372 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b97968-88c1-4c3d-810e-5ccdcc4820e5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:37.230286 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950921be-3e73-4847-b7a5-97476cc94b46 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:37.245106 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69f57a-302e-43f8-a007-86ac2cd5eedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:37.245774 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69f57a-302e-43f8-a007-86ac2cd5eedf None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:37.246052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69f57a-302e-43f8-a007-86ac2cd5eedf None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:37.246318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69f57a-302e-43f8-a007-86ac2cd5eedf None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:37.247022 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69f57a-302e-43f8-a007-86ac2cd5eedf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:37.247324 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69f57a-302e-43f8-a007-86ac2cd5eedf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:37.248703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69f57a-302e-43f8-a007-86ac2cd5eedf None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:37.254019 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 508/1003] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Jun 25 15:54:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:37.260467 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69f57a-302e-43f8-a007-86ac2cd5eedf None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:37.262518 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:37.269765 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad362a-9082-458d-99b7-6b801fd3b81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:37.270427 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad362a-9082-458d-99b7-6b801fd3b81b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:37.270696 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad362a-9082-458d-99b7-6b801fd3b81b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:37.270965 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad362a-9082-458d-99b7-6b801fd3b81b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:37.272181 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad362a-9082-458d-99b7-6b801fd3b81b None admin] RBAC: Adding query filter params (interface=public, service_id=0b3b6047df0c437b96f0dd7c8e883659, region_id=RegionOne) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:37.272525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad362a-9082-458d-99b7-6b801fd3b81b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:37.274344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad362a-9082-458d-99b7-6b801fd3b81b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:37.280456 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 496/1004] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Jun 25 15:54:37 2022] GET /identity/v3/endpoints?service_id=0b3b6047df0c437b96f0dd7c8e883659&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:37.285921 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad362a-9082-458d-99b7-6b801fd3b81b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:37.300862 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:37.381599 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69f57a-302e-43f8-a007-86ac2cd5eedf None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:37.389375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad362a-9082-458d-99b7-6b801fd3b81b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:37.418735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b7e72-31cc-4225-b5ae-22943ec702bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:37.419484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b7e72-31cc-4225-b5ae-22943ec702bf None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:37.419692 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b7e72-31cc-4225-b5ae-22943ec702bf None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:37.419966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b7e72-31cc-4225-b5ae-22943ec702bf None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:37.420811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b7e72-31cc-4225-b5ae-22943ec702bf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:37.421176 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b7e72-31cc-4225-b5ae-22943ec702bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:37.422554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b7e72-31cc-4225-b5ae-22943ec702bf None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:37.426295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740698af-fd0b-4a25-a486-7bc00a676c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:37.426959 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740698af-fd0b-4a25-a486-7bc00a676c17 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:37.427239 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740698af-fd0b-4a25-a486-7bc00a676c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:37.427510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740698af-fd0b-4a25-a486-7bc00a676c17 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:37.432519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 509/1005] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Jun 25 15:54:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:37.432652 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740698af-fd0b-4a25-a486-7bc00a676c17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:37.432652 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740698af-fd0b-4a25-a486-7bc00a676c17 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:37.433331 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b7e72-31cc-4225-b5ae-22943ec702bf None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:37.438090 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 497/1006] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Jun 25 15:54:37 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 15:54:37.544673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:37.553147 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b7e72-31cc-4225-b5ae-22943ec702bf None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:37.591861 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afeea9-1842-4e8a-9c12-c948ce24d02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:37.592526 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afeea9-1842-4e8a-9c12-c948ce24d02c None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:37.592812 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afeea9-1842-4e8a-9c12-c948ce24d02c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:37.593086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afeea9-1842-4e8a-9c12-c948ce24d02c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:37.593865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0afeea9-1842-4e8a-9c12-c948ce24d02c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:37.595235 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0afeea9-1842-4e8a-9c12-c948ce24d02c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:37.612537 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 510/1007] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Jun 25 15:54:37 2022] POST /identity/v3/endpoints => generated 325 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 15:54:38.813416 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d118099-3481-455e-83d0-39ce48fa73db None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:38.813966 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d118099-3481-455e-83d0-39ce48fa73db None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:38.813966 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d118099-3481-455e-83d0-39ce48fa73db None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:38.815467 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 498/1008] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:38.820539 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07dcddf-56c0-4459-9c73-11fec6465d1e None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:38.820981 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07dcddf-56c0-4459-9c73-11fec6465d1e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:38.821061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07dcddf-56c0-4459-9c73-11fec6465d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:38.840772 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a07dcddf-56c0-4459-9c73-11fec6465d1e None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:38.841953 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a07dcddf-56c0-4459-9c73-11fec6465d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:38.924337 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 511/1009] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:38 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:38.948449 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d118099-3481-455e-83d0-39ce48fa73db None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:39.027874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:39.072038 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d118099-3481-455e-83d0-39ce48fa73db None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:39.107003 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b7047-bf2d-4147-9c11-306272d03f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:39.107666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b7047-bf2d-4147-9c11-306272d03f51 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:39.107943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b7047-bf2d-4147-9c11-306272d03f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:39.108209 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b7047-bf2d-4147-9c11-306272d03f51 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:39.108952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b7047-bf2d-4147-9c11-306272d03f51 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:39.110326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b7047-bf2d-4147-9c11-306272d03f51 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:39.125441 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 499/1010] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Jun 25 15:54:38 2022] POST /identity/v3/services => generated 242 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:54:39.463154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705de65-47e9-4b51-be4a-bf185a545b8b None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:39.463492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705de65-47e9-4b51-be4a-bf185a545b8b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:39.463801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705de65-47e9-4b51-be4a-bf185a545b8b None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:39.465374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 512/1011] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:39.470703 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718ea2b-7bb9-4479-bfdb-840059b191b8 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:39.470960 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718ea2b-7bb9-4479-bfdb-840059b191b8 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:39.471251 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718ea2b-7bb9-4479-bfdb-840059b191b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:39.488287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a718ea2b-7bb9-4479-bfdb-840059b191b8 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:39.489499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a718ea2b-7bb9-4479-bfdb-840059b191b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:39.565371 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 500/1012] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:39 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:39.588870 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705de65-47e9-4b51-be4a-bf185a545b8b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:39.645545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac4b4c-ff48-4004-b66d-6ad0efe596e0 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:39.645883 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac4b4c-ff48-4004-b66d-6ad0efe596e0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:39.646041 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac4b4c-ff48-4004-b66d-6ad0efe596e0 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:39.647433 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 501/1013] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:54:39.652890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fccdaf-083c-49bb-ae97-bc189c8b0314 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:39.653185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fccdaf-083c-49bb-ae97-bc189c8b0314 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:39.653447 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fccdaf-083c-49bb-ae97-bc189c8b0314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:39.677267 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-36fccdaf-083c-49bb-ae97-bc189c8b0314 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:39.678450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-36fccdaf-083c-49bb-ae97-bc189c8b0314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:39.683661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:39.709363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705de65-47e9-4b51-be4a-bf185a545b8b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:39.719561 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:39.747375 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b432bff-a1ab-430d-9cbc-dd01fa26f3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:39.748179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b432bff-a1ab-430d-9cbc-dd01fa26f3cc None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:39.748683 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 502/1014] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:39 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:39.748819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b432bff-a1ab-430d-9cbc-dd01fa26f3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:39.748819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b432bff-a1ab-430d-9cbc-dd01fa26f3cc None admin] PATH_INFO: `/v3/services/compute` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:39.749405 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b432bff-a1ab-430d-9cbc-dd01fa26f3cc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:39.750840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b432bff-a1ab-430d-9cbc-dd01fa26f3cc None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:39.763335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-7b432bff-a1ab-430d-9cbc-dd01fa26f3cc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 25 15:54:39.763335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:39.763335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:39.763335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:39.763335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:39.763335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:39.763335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:39.763335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:39.763335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 25 15:54:39.763335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:39.763335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:39.763335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:39.763335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:39.763335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:39.763335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:39.763335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:39.763335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:39.764374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:39.764374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:39.764374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:39.764374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:39.764374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:39.764374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:39.764374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:39.764374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:39.764374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:39.764374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:39.764374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:39.764374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:39.764374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:39.764374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:39.764374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:39.764374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:39.765947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:39.765947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:39.765947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:39.765947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:39.765947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:39.765947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:39.765947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:39.765947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:39.765947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:39.765947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:39.765947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:39.765947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:39.765947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 25 15:54:39.765947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:39.765947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 513/1015] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:39.773836 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fccdaf-083c-49bb-ae97-bc189c8b0314 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:39.776183 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b432bff-a1ab-430d-9cbc-dd01fa26f3cc None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:39.880374 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fccdaf-083c-49bb-ae97-bc189c8b0314 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:39.889720 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b432bff-a1ab-430d-9cbc-dd01fa26f3cc None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:39.921737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33a7d9-d73d-4eea-aea2-4c7f0ab8c2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:39.922412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33a7d9-d73d-4eea-aea2-4c7f0ab8c2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:39.922695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33a7d9-d73d-4eea-aea2-4c7f0ab8c2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:39.922865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5ecb4-c4e8-4a7a-81f8-492682bc36b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:39.922973 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33a7d9-d73d-4eea-aea2-4c7f0ab8c2a6 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:39.923650 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5ecb4-c4e8-4a7a-81f8-492682bc36b7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:39.923771 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33a7d9-d73d-4eea-aea2-4c7f0ab8c2a6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:39.923993 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5ecb4-c4e8-4a7a-81f8-492682bc36b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:39.924090 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33a7d9-d73d-4eea-aea2-4c7f0ab8c2a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:39.924245 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5ecb4-c4e8-4a7a-81f8-492682bc36b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:39.925376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5ecb4-c4e8-4a7a-81f8-492682bc36b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:39.925473 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33a7d9-d73d-4eea-aea2-4c7f0ab8c2a6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:39.926788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5ecb4-c4e8-4a7a-81f8-492682bc36b7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:39.933420 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 514/1016] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:39.933554 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 503/1017] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Jun 25 15:54:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:39.936233 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5ecb4-c4e8-4a7a-81f8-492682bc36b7 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:39.936358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33a7d9-d73d-4eea-aea2-4c7f0ab8c2a6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:39.954259 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:39.979203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:40.045715 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5ecb4-c4e8-4a7a-81f8-492682bc36b7 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:40.048539 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33a7d9-d73d-4eea-aea2-4c7f0ab8c2a6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:40.081805 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a08c30-5188-4ec2-96ae-d45f6a3c02ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:40.082471 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a08c30-5188-4ec2-96ae-d45f6a3c02ee None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:40.082754 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a08c30-5188-4ec2-96ae-d45f6a3c02ee None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:40.082853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54fbc2-919f-4e67-8029-f835aeb84e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:40.083024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a08c30-5188-4ec2-96ae-d45f6a3c02ee None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:40.083482 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54fbc2-919f-4e67-8029-f835aeb84e17 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:40.083734 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a08c30-5188-4ec2-96ae-d45f6a3c02ee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:40.083841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54fbc2-919f-4e67-8029-f835aeb84e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:40.084025 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54fbc2-919f-4e67-8029-f835aeb84e17 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:40.084766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54fbc2-919f-4e67-8029-f835aeb84e17 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:40.085072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54fbc2-919f-4e67-8029-f835aeb84e17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:40.085239 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a08c30-5188-4ec2-96ae-d45f6a3c02ee None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:40.086507 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54fbc2-919f-4e67-8029-f835aeb84e17 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:40.095987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 504/1018] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Jun 25 15:54:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:40.096900 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54fbc2-919f-4e67-8029-f835aeb84e17 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:40.117847 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 515/1019] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Jun 25 15:54:39 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:54:40.209194 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54fbc2-919f-4e67-8029-f835aeb84e17 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:40.210998 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:40.253084 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776b6a5-8e4e-46ef-a9c8-0632c037ea3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:40.253661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776b6a5-8e4e-46ef-a9c8-0632c037ea3d None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:40.253938 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776b6a5-8e4e-46ef-a9c8-0632c037ea3d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:40.254205 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776b6a5-8e4e-46ef-a9c8-0632c037ea3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:40.254955 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8776b6a5-8e4e-46ef-a9c8-0632c037ea3d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:40.256340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8776b6a5-8e4e-46ef-a9c8-0632c037ea3d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:40.278825 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 505/1020] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Jun 25 15:54:40 2022] POST /identity/v3/endpoints => generated 334 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:54:41.151594 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f25b2e-73e4-4ee6-8587-9bf2ccf18259 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:41.152201 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f25b2e-73e4-4ee6-8587-9bf2ccf18259 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:41.152201 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f25b2e-73e4-4ee6-8587-9bf2ccf18259 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:41.153453 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 516/1021] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:41.158858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55c7ca-771f-4e4b-9bb8-784bd84c86da None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:41.159264 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55c7ca-771f-4e4b-9bb8-784bd84c86da None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:41.159358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55c7ca-771f-4e4b-9bb8-784bd84c86da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:41.177025 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-8f55c7ca-771f-4e4b-9bb8-784bd84c86da None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:41.178218 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8f55c7ca-771f-4e4b-9bb8-784bd84c86da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:41.253904 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 506/1022] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:41.277691 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f25b2e-73e4-4ee6-8587-9bf2ccf18259 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:41.346056 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:41.386695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f25b2e-73e4-4ee6-8587-9bf2ccf18259 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:41.420358 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047a842-5c57-4565-9288-6f405339dc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:41.421113 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047a842-5c57-4565-9288-6f405339dc37 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:41.421394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047a842-5c57-4565-9288-6f405339dc37 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:41.421668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047a842-5c57-4565-9288-6f405339dc37 None admin] PATH_INFO: `/v3/services/image` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:41.422325 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c047a842-5c57-4565-9288-6f405339dc37 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:41.423692 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c047a842-5c57-4565-9288-6f405339dc37 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:41.435304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-c047a842-5c57-4565-9288-6f405339dc37 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 25 15:54:41.435304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:41.435304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:41.435304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:41.435304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:41.435304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:41.435304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:41.435304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:41.435304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 25 15:54:41.435304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:41.435304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:41.435304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:41.435304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:41.435304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:41.435304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:41.435304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:41.435304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:41.436316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:41.436316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:41.436316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:41.436316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:41.436316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:41.436316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:41.436316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:41.436316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:41.436316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:41.436316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:41.436316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:41.436316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:41.436316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:41.436316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:41.436316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:41.436316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:41.437969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:41.437969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:41.437969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:41.437969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:41.437969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:41.437969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:41.437969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:41.437969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:41.437969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:41.437969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:41.437969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:41.437969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:41.437969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 25 15:54:41.437969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:41.437969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 517/1023] 10.222.0.13 () {56 vars in 1105 bytes} [Sat Jun 25 15:54:41 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:41.442412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f55c7ca-771f-4e4b-9bb8-784bd84c86da None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:41.533774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:41.552372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f55c7ca-771f-4e4b-9bb8-784bd84c86da None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:41.587799 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf10a0e-8d6e-4272-aa6a-b7da1c2d0bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:41.588635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf10a0e-8d6e-4272-aa6a-b7da1c2d0bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:41.588896 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf10a0e-8d6e-4272-aa6a-b7da1c2d0bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:41.589169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf10a0e-8d6e-4272-aa6a-b7da1c2d0bd2 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:41.589911 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf10a0e-8d6e-4272-aa6a-b7da1c2d0bd2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:41.590244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf10a0e-8d6e-4272-aa6a-b7da1c2d0bd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:41.591839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf10a0e-8d6e-4272-aa6a-b7da1c2d0bd2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:41.602704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 507/1024] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:41.603772 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047a842-5c57-4565-9288-6f405339dc37 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:41.708237 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047a842-5c57-4565-9288-6f405339dc37 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:41.734751 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:41.744075 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823e880-493f-44e8-a4e6-2276b3a97fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:41.744774 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823e880-493f-44e8-a4e6-2276b3a97fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:41.745050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823e880-493f-44e8-a4e6-2276b3a97fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:41.745321 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823e880-493f-44e8-a4e6-2276b3a97fa0 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:41.746028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823e880-493f-44e8-a4e6-2276b3a97fa0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:41.746327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823e880-493f-44e8-a4e6-2276b3a97fa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:41.747686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823e880-493f-44e8-a4e6-2276b3a97fa0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:41.752897 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 518/1025] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:41.758792 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf10a0e-8d6e-4272-aa6a-b7da1c2d0bd2 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:41.872068 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf10a0e-8d6e-4272-aa6a-b7da1c2d0bd2 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:41.909915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3f4ff-aa8e-45fd-868c-53e13c1516bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:41.910981 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3f4ff-aa8e-45fd-868c-53e13c1516bd None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:41.911457 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3f4ff-aa8e-45fd-868c-53e13c1516bd None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:41.911919 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3f4ff-aa8e-45fd-868c-53e13c1516bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:41.913157 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3f4ff-aa8e-45fd-868c-53e13c1516bd None admin] RBAC: Adding query filter params (interface=public, service_id=7b63c1ef5c924fedb517c5744100e78f, region_id=RegionOne) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:41.913757 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3f4ff-aa8e-45fd-868c-53e13c1516bd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:41.915684 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3f4ff-aa8e-45fd-868c-53e13c1516bd None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:41.921921 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 508/1026] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Jun 25 15:54:41 2022] GET /identity/v3/endpoints?service_id=7b63c1ef5c924fedb517c5744100e78f&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:41.927704 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823e880-493f-44e8-a4e6-2276b3a97fa0 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:42.036402 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823e880-493f-44e8-a4e6-2276b3a97fa0 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:42.072517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfaacb-7917-4647-a74d-e2abbd8f9f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:42.073184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfaacb-7917-4647-a74d-e2abbd8f9f9d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:42.073461 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfaacb-7917-4647-a74d-e2abbd8f9f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:42.073732 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfaacb-7917-4647-a74d-e2abbd8f9f9d None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:42.074412 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebfaacb-7917-4647-a74d-e2abbd8f9f9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:42.075774 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebfaacb-7917-4647-a74d-e2abbd8f9f9d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:42.082671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 519/1027] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Jun 25 15:54:41 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 15:54:42.202151 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1674ef9-4eb1-4aaa-86e0-bad8533642a8 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:42.202673 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1674ef9-4eb1-4aaa-86e0-bad8533642a8 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:42.202673 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1674ef9-4eb1-4aaa-86e0-bad8533642a8 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:42.204448 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 509/1028] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:42.209969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eccb74-7d91-4428-b5d0-0d6ab00df329 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:42.210609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eccb74-7d91-4428-b5d0-0d6ab00df329 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:42.210684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eccb74-7d91-4428-b5d0-0d6ab00df329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:42.228741 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-02eccb74-7d91-4428-b5d0-0d6ab00df329 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:42.229943 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-02eccb74-7d91-4428-b5d0-0d6ab00df329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:42.285590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:42.303434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 520/1029] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:42.328960 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1674ef9-4eb1-4aaa-86e0-bad8533642a8 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:42.345066 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:42.421253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55614887-72fa-4f53-9a58-4074831f1747 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:42.421602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55614887-72fa-4f53-9a58-4074831f1747 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:42.421812 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55614887-72fa-4f53-9a58-4074831f1747 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:42.423336 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 521/1030] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:54:42.428060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f685c68-5a68-43d2-986e-4b7c638793c1 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:42.428351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f685c68-5a68-43d2-986e-4b7c638793c1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:42.428671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f685c68-5a68-43d2-986e-4b7c638793c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:42.437440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1674ef9-4eb1-4aaa-86e0-bad8533642a8 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:42.445789 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5f685c68-5a68-43d2-986e-4b7c638793c1 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:42.446986 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5f685c68-5a68-43d2-986e-4b7c638793c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:42.473580 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bf19d-1b37-485f-812f-ae3dee1864c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:42.474622 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bf19d-1b37-485f-812f-ae3dee1864c1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:42.475042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bf19d-1b37-485f-812f-ae3dee1864c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:42.475454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74bf19d-1b37-485f-812f-ae3dee1864c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:42.477006 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74bf19d-1b37-485f-812f-ae3dee1864c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:42.478801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74bf19d-1b37-485f-812f-ae3dee1864c1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:42.485470 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 510/1031] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:42.486452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bf19d-1b37-485f-812f-ae3dee1864c1 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:42.514803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 522/1032] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:42.539346 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f685c68-5a68-43d2-986e-4b7c638793c1 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:42.569521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:42.592049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74bf19d-1b37-485f-812f-ae3dee1864c1 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:42.636913 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dd801-8647-47d3-9408-db56317e5f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:42.637594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dd801-8647-47d3-9408-db56317e5f85 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:42.637878 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dd801-8647-47d3-9408-db56317e5f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:42.638151 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dd801-8647-47d3-9408-db56317e5f85 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:42.638899 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793dd801-8647-47d3-9408-db56317e5f85 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:42.640310 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793dd801-8647-47d3-9408-db56317e5f85 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:42.646950 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f685c68-5a68-43d2-986e-4b7c638793c1 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:42.672441 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 511/1033] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Jun 25 15:54:42 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:54:42.677028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:42.699293 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f2b21-4d11-4010-9b39-ba6c05a63d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:42.700131 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f2b21-4d11-4010-9b39-ba6c05a63d9c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:42.700438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f2b21-4d11-4010-9b39-ba6c05a63d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:42.700730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f2b21-4d11-4010-9b39-ba6c05a63d9c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:42.712531 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f2b21-4d11-4010-9b39-ba6c05a63d9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:54:42.721222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-f57f2b21-4d11-4010-9b39-ba6c05a63d9c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 25 15:54:42.721222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:42.721222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:42.721222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:42.721222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:42.721222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:42.721222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:42.721222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:42.721222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:42.721222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:42.721222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:42.721222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:42.721222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:54:42.721222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:54:42.721222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:54:42.721222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:54:42.722234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:54:42.722234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:54:42.722234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:54:42.722234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:54:42.722234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:42.722234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:42.722234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:42.722234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:42.722234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:42.722234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:42.722234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:42.722234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:42.722234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:42.722234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:42.722234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:42.722234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:42.723079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:42.723079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:42.723079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:42.723079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:42.723079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:54:42.723079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:54:42.723079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:54:42.723079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:54:42.723079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:54:42.723079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:54:42.723079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 25 15:54:42.723079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:42.723079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 523/1034] 10.222.0.13 () {56 vars in 1120 bytes} [Sat Jun 25 15:54:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:42.728745 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dd801-8647-47d3-9408-db56317e5f85 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:42.839954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dd801-8647-47d3-9408-db56317e5f85 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:42.880400 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4d4c6-53f3-49b7-89c4-99f4e4333f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:42.881100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4d4c6-53f3-49b7-89c4-99f4e4333f94 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:42.881383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4d4c6-53f3-49b7-89c4-99f4e4333f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:42.881648 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4d4c6-53f3-49b7-89c4-99f4e4333f94 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:42.882589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4d4c6-53f3-49b7-89c4-99f4e4333f94 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:42.883025 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4d4c6-53f3-49b7-89c4-99f4e4333f94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:42.884434 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4d4c6-53f3-49b7-89c4-99f4e4333f94 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:42.886756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:42.899821 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 512/1035] 10.222.0.13 () {56 vars in 1115 bytes} [Sat Jun 25 15:54:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:42.900834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f2b21-4d11-4010-9b39-ba6c05a63d9c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:43.003472 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f2b21-4d11-4010-9b39-ba6c05a63d9c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:43.039416 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec4396-dea8-4823-975c-c5b076ba4e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:43.040540 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec4396-dea8-4823-975c-c5b076ba4e4e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:43.041003 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec4396-dea8-4823-975c-c5b076ba4e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:43.041473 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec4396-dea8-4823-975c-c5b076ba4e4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:43.053080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec4396-dea8-4823-975c-c5b076ba4e4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:43.054818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec4396-dea8-4823-975c-c5b076ba4e4e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:43.072085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-75ec4396-dea8-4823-975c-c5b076ba4e4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:43.072085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:43.072085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:43.072085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:43.072085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:43.072085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:43.072085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:43.072085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:43.072085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:43.072085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:43.072085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:43.072085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:43.072085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:43.072085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:43.072085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:43.073061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:43.073061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:43.073061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:43.073061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:43.073061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:43.073061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:43.073061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:43.073061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:43.073061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:43.073061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:43.073061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:43.073061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:43.073061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:43.073061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:43.073061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:43.073061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:43.073061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:43.073985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:43.073985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:43.073985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:43.073985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:43.074278 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 524/1036] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:43.080337 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4d4c6-53f3-49b7-89c4-99f4e4333f94 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:43.186440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4d4c6-53f3-49b7-89c4-99f4e4333f94 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:43.223515 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371602d0-76ed-4348-94e8-497183b58e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:43.224142 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371602d0-76ed-4348-94e8-497183b58e05 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:43.224428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371602d0-76ed-4348-94e8-497183b58e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:43.224730 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371602d0-76ed-4348-94e8-497183b58e05 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:43.225451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371602d0-76ed-4348-94e8-497183b58e05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:43.225756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371602d0-76ed-4348-94e8-497183b58e05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:43.227409 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371602d0-76ed-4348-94e8-497183b58e05 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:43.239018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 513/1037] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:43.244963 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec4396-dea8-4823-975c-c5b076ba4e4e None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:43.274978 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:43.349695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec4396-dea8-4823-975c-c5b076ba4e4e None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:43.383919 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e151e54-09e6-4077-a4bd-c8c24e126e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:43.384737 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e151e54-09e6-4077-a4bd-c8c24e126e66 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:43.385018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e151e54-09e6-4077-a4bd-c8c24e126e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:43.385282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e151e54-09e6-4077-a4bd-c8c24e126e66 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:43.393133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e151e54-09e6-4077-a4bd-c8c24e126e66 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:43.394554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e151e54-09e6-4077-a4bd-c8c24e126e66 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:43.408274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-6e151e54-09e6-4077-a4bd-c8c24e126e66 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 25 15:54:43.408274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:43.408274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:43.408274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:43.408274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:43.408274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:43.408274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:43.408274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:43.408274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:43.408274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:43.408274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:43.408274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:43.408274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:54:43.408274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:54:43.408274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:54:43.408274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:54:43.409250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:43.409250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:43.409250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:54:43.409250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:43.409250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:54:43.409250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:43.409250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:43.409250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:43.409250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:43.409250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:43.409250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:43.409250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:43.409250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:43.409250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:43.409250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:43.409250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:43.409250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:43.409250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:43.410204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:43.410204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:43.410204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:54:43.410204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:54:43.410204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:54:43.410204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:54:43.410204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:54:43.410204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:54:43.410204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 25 15:54:43.410204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:43.412518 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 525/1038] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Jun 25 15:54:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:43.415565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371602d0-76ed-4348-94e8-497183b58e05 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:43.478348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:43.522784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371602d0-76ed-4348-94e8-497183b58e05 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:43.556302 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3a6d6-78c4-4805-93e3-2d06bfe73f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:43.557003 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3a6d6-78c4-4805-93e3-2d06bfe73f1b None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:43.557281 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3a6d6-78c4-4805-93e3-2d06bfe73f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:43.557550 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3a6d6-78c4-4805-93e3-2d06bfe73f1b None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:43.558382 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d3a6d6-78c4-4805-93e3-2d06bfe73f1b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:43.558683 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d3a6d6-78c4-4805-93e3-2d06bfe73f1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:43.560116 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d3a6d6-78c4-4805-93e3-2d06bfe73f1b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:43.577633 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 514/1039] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Jun 25 15:54:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:43.583295 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e151e54-09e6-4077-a4bd-c8c24e126e66 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:43.672760 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:43.691071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e151e54-09e6-4077-a4bd-c8c24e126e66 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:43.725826 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098cf83-86f4-4c48-88d0-03aa207633eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:43.726517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098cf83-86f4-4c48-88d0-03aa207633eb None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:43.726791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098cf83-86f4-4c48-88d0-03aa207633eb None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:43.727065 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098cf83-86f4-4c48-88d0-03aa207633eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:43.793448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098cf83-86f4-4c48-88d0-03aa207633eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:43.794796 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7098cf83-86f4-4c48-88d0-03aa207633eb None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:43.890353 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:43.904154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 526/1040] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:54:43 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:54:43.910054 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3a6d6-78c4-4805-93e3-2d06bfe73f1b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:44.012478 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3a6d6-78c4-4805-93e3-2d06bfe73f1b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:44.029026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:44.047794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d25b2-b818-40ed-a89f-cf38486e9195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:44.048545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d25b2-b818-40ed-a89f-cf38486e9195 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:44.048824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d25b2-b818-40ed-a89f-cf38486e9195 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:44.049089 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d25b2-b818-40ed-a89f-cf38486e9195 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:44.059410 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d25b2-b818-40ed-a89f-cf38486e9195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:44.060880 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d25b2-b818-40ed-a89f-cf38486e9195 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:44.078885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-5b3d25b2-b818-40ed-a89f-cf38486e9195 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:44.078885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:44.078885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:44.078885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:44.078885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:44.078885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:44.078885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:44.078885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:44.078885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:44.078885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:44.078885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:44.078885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:44.078885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:44.078885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:44.078885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:44.079830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:44.079830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:44.079830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:44.079830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:44.079830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:44.079830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:44.079830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:44.079830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:44.079830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:44.079830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:44.079830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:44.079830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:44.079830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:44.079830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:44.079830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:44.079830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:44.079830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:44.081394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:44.081394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:44.081394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:44.081394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:44.081898 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 515/1041] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:44.085933 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098cf83-86f4-4c48-88d0-03aa207633eb None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:44.107664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a047e-1c8c-4bd0-9e53-cef86f3d5403 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:44.107972 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a047e-1c8c-4bd0-9e53-cef86f3d5403 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:44.108213 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a047e-1c8c-4bd0-9e53-cef86f3d5403 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:44.109715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 516/1042] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:54:44.114691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78920b-ed87-4f08-93b0-a3d08b2c91a0 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:44.115081 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78920b-ed87-4f08-93b0-a3d08b2c91a0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:44.115353 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78920b-ed87-4f08-93b0-a3d08b2c91a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:44.133156 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ba78920b-ed87-4f08-93b0-a3d08b2c91a0 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:44.134850 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ba78920b-ed87-4f08-93b0-a3d08b2c91a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:44.190657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7098cf83-86f4-4c48-88d0-03aa207633eb None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:44.205140 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 517/1043] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:44.228331 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78920b-ed87-4f08-93b0-a3d08b2c91a0 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:44.230383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e449f384-08fd-4b0a-8784-2a8a2723b69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:44.231077 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449f384-08fd-4b0a-8784-2a8a2723b69d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:44.231352 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449f384-08fd-4b0a-8784-2a8a2723b69d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:44.231586 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449f384-08fd-4b0a-8784-2a8a2723b69d None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:44.232306 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e449f384-08fd-4b0a-8784-2a8a2723b69d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:44.232665 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e449f384-08fd-4b0a-8784-2a8a2723b69d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:44.234050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e449f384-08fd-4b0a-8784-2a8a2723b69d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:44.246621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 527/1044] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:44.251946 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e449f384-08fd-4b0a-8784-2a8a2723b69d None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:44.303496 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:44.328756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:44.332537 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78920b-ed87-4f08-93b0-a3d08b2c91a0 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:44.361953 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e449f384-08fd-4b0a-8784-2a8a2723b69d None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:44.371011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dc991-a79e-4e1b-aca7-262fb7c44120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:44.371821 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dc991-a79e-4e1b-aca7-262fb7c44120 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:44.372097 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dc991-a79e-4e1b-aca7-262fb7c44120 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:44.372369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dc991-a79e-4e1b-aca7-262fb7c44120 None admin] PATH_INFO: `/v3/services/image` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:44.373053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355dc991-a79e-4e1b-aca7-262fb7c44120 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:44.374436 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355dc991-a79e-4e1b-aca7-262fb7c44120 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:44.386180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-355dc991-a79e-4e1b-aca7-262fb7c44120 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 25 15:54:44.386180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:44.386180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:54:44.386180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:54:44.386180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:54:44.386180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:54:44.386180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:54:44.386180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:44.386180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 25 15:54:44.386180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:44.386180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:54:44.386180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:44.386180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:44.386180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:44.386180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:44.386180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:44.386180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:44.387444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:44.387444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:44.387444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:44.387444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:44.387444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:44.387444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:44.387444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:54:44.387444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:54:44.387444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:54:44.387444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:54:44.387444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:44.387444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:44.387444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:44.387444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:44.387444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:44.387444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:44.389026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:44.389026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:44.389026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:44.389026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:44.389026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:44.389026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:44.389026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:44.389026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:44.389026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:44.389026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:44.389026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:54:44.389026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:54:44.389026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 25 15:54:44.389026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:44.392709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 518/1045] 10.222.0.13 () {56 vars in 1105 bytes} [Sat Jun 25 15:54:44 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:44.393538 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dc991-a79e-4e1b-aca7-262fb7c44120 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:44.402067 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73267f82-f9f5-4586-aa45-7447a7c60566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:44.402871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73267f82-f9f5-4586-aa45-7447a7c60566 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:44.403153 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73267f82-f9f5-4586-aa45-7447a7c60566 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:44.403432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73267f82-f9f5-4586-aa45-7447a7c60566 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:44.411937 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73267f82-f9f5-4586-aa45-7447a7c60566 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:44.413432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73267f82-f9f5-4586-aa45-7447a7c60566 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:44.428137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-73267f82-f9f5-4586-aa45-7447a7c60566 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:44.428137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:44.428137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:44.428137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:44.428137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:44.428137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:44.428137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:44.428137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:44.428137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:44.428137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:44.428137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:44.428137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:44.428137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:54:44.428137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:54:44.428137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:54:44.429212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:54:44.429212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:44.429212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:44.429212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:44.429212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:44.429212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:44.429212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:44.429212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:44.429212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:44.429212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:44.429212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:44.429212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:44.429212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:44.429212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:44.429212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:44.429212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:44.429212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:44.430795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:54:44.430795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:54:44.430795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:54:44.430795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:54:44.430795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:54:44.430795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:54:44.430795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:44.430795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:44.430795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 528/1046] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:44.435036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73267f82-f9f5-4586-aa45-7447a7c60566 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:44.500840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dc991-a79e-4e1b-aca7-262fb7c44120 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:44.538637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e48dfc-80c2-4048-9de4-0c1b0b5f7b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:44.539308 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e48dfc-80c2-4048-9de4-0c1b0b5f7b0c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:44.539588 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e48dfc-80c2-4048-9de4-0c1b0b5f7b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:44.539858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e48dfc-80c2-4048-9de4-0c1b0b5f7b0c None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:44.540606 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e48dfc-80c2-4048-9de4-0c1b0b5f7b0c None admin] RBAC: Adding query filter params (name=image) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:44.540824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73267f82-f9f5-4586-aa45-7447a7c60566 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:44.540937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e48dfc-80c2-4048-9de4-0c1b0b5f7b0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:44.542296 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e48dfc-80c2-4048-9de4-0c1b0b5f7b0c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:44.547362 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 519/1047] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:44.552886 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e48dfc-80c2-4048-9de4-0c1b0b5f7b0c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:44.554824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:44.581845 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:44.583138 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d35f48-e009-405a-b22a-e98418009b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:44.583996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d35f48-e009-405a-b22a-e98418009b81 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:44.584307 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d35f48-e009-405a-b22a-e98418009b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:44.584614 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d35f48-e009-405a-b22a-e98418009b81 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:44.585791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d35f48-e009-405a-b22a-e98418009b81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:44.586479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d35f48-e009-405a-b22a-e98418009b81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:44.587570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d35f48-e009-405a-b22a-e98418009b81 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:44.606194 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 529/1048] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:54:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:44.607888 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d35f48-e009-405a-b22a-e98418009b81 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:44.664591 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e48dfc-80c2-4048-9de4-0c1b0b5f7b0c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:44.698275 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc4d36-4cb3-4d73-820a-0bc5d2b1ec60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:44.698940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc4d36-4cb3-4d73-820a-0bc5d2b1ec60 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:44.699217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc4d36-4cb3-4d73-820a-0bc5d2b1ec60 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:44.699487 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc4d36-4cb3-4d73-820a-0bc5d2b1ec60 None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:44.700190 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc4d36-4cb3-4d73-820a-0bc5d2b1ec60 None admin] RBAC: Adding query filter params (type=image) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:44.700529 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc4d36-4cb3-4d73-820a-0bc5d2b1ec60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:44.701900 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc4d36-4cb3-4d73-820a-0bc5d2b1ec60 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:44.711320 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 520/1049] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:44.712269 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc4d36-4cb3-4d73-820a-0bc5d2b1ec60 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:44.719240 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d35f48-e009-405a-b22a-e98418009b81 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:44.758172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b27e3-ee5c-4f9d-b13f-a1b2ba81bfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:44.758830 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b27e3-ee5c-4f9d-b13f-a1b2ba81bfc1 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:44.759113 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b27e3-ee5c-4f9d-b13f-a1b2ba81bfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:44.759387 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b27e3-ee5c-4f9d-b13f-a1b2ba81bfc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:44.760233 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b27e3-ee5c-4f9d-b13f-a1b2ba81bfc1 None admin] RBAC: Adding query filter params (role.id=c38990a9778e4b2abd3376f9e066d3a5, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=3622c5ccb7534e7c8e8342b4f6643bc3) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:44.760555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b27e3-ee5c-4f9d-b13f-a1b2ba81bfc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:44.761971 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b27e3-ee5c-4f9d-b13f-a1b2ba81bfc1 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:44.767780 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 530/1050] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:54:44 2022] GET /identity/v3/role_assignments?user.id=3622c5ccb7534e7c8e8342b4f6643bc3&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=c38990a9778e4b2abd3376f9e066d3a5 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:44.776576 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714f93d-c7f6-4615-90d1-4bca9ec2fe8e None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:44.776915 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714f93d-c7f6-4615-90d1-4bca9ec2fe8e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:44.777190 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714f93d-c7f6-4615-90d1-4bca9ec2fe8e None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:44.778661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 531/1051] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:54:44.784456 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cd357-59e2-4fc3-8c0e-2f9646ed78aa None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:44.784722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cd357-59e2-4fc3-8c0e-2f9646ed78aa None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:44.784996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1cd357-59e2-4fc3-8c0e-2f9646ed78aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:44.802191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-2c1cd357-59e2-4fc3-8c0e-2f9646ed78aa None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:44.803409 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2c1cd357-59e2-4fc3-8c0e-2f9646ed78aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:44.823123 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:44.834632 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:44.835359 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc4d36-4cb3-4d73-820a-0bc5d2b1ec60 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:44.880198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e19e4-aba8-48c6-afed-d50422d7dd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:44.881010 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e19e4-aba8-48c6-afed-d50422d7dd4b None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:44.881292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e19e4-aba8-48c6-afed-d50422d7dd4b None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:44.881574 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e19e4-aba8-48c6-afed-d50422d7dd4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:44.882506 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e19e4-aba8-48c6-afed-d50422d7dd4b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:44.884080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e19e4-aba8-48c6-afed-d50422d7dd4b None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:44.884731 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 532/1052] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:44.908855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1cd357-59e2-4fc3-8c0e-2f9646ed78aa None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:44.920451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 521/1053] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Jun 25 15:54:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 15:54:45.028835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1cd357-59e2-4fc3-8c0e-2f9646ed78aa None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:45.074036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e163500-5aeb-4593-83f9-5b25f9ccac01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:45.075176 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e163500-5aeb-4593-83f9-5b25f9ccac01 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:45.075631 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e163500-5aeb-4593-83f9-5b25f9ccac01 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:45.075924 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e163500-5aeb-4593-83f9-5b25f9ccac01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:45.086746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e163500-5aeb-4593-83f9-5b25f9ccac01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:54:45.094283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-5e163500-5aeb-4593-83f9-5b25f9ccac01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:54:45.094283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:45.094283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:45.094283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:45.094283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:45.094283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:45.094283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:45.094283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:45.094283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:45.094283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:45.094283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:45.094283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:45.094283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:54:45.094283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:54:45.094283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:54:45.094283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:54:45.095444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:54:45.095444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:54:45.095444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:54:45.095444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:54:45.095444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:45.095444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:45.095444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:45.095444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:45.095444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:45.095444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:45.095444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:45.095444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:45.095444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:45.095444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:45.095444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:45.095444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:45.096591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:45.096591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:45.096591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:45.096591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:45.096591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:54:45.096591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:54:45.096591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:54:45.096591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:54:45.096591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:54:45.096591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:54:45.096591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:54:45.096591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:45.096591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 533/1054] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:54:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:45.102434 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e19e4-aba8-48c6-afed-d50422d7dd4b None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:45.205777 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e19e4-aba8-48c6-afed-d50422d7dd4b None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:45.239875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef003b0-f383-47c9-8b3a-95bbdcd78cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:45.240650 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef003b0-f383-47c9-8b3a-95bbdcd78cee None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:45.240886 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef003b0-f383-47c9-8b3a-95bbdcd78cee None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:45.241162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef003b0-f383-47c9-8b3a-95bbdcd78cee None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:45.242008 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef003b0-f383-47c9-8b3a-95bbdcd78cee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:45.242304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef003b0-f383-47c9-8b3a-95bbdcd78cee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:45.243684 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef003b0-f383-47c9-8b3a-95bbdcd78cee None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:45.257268 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 522/1055] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:54:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:45.258843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e163500-5aeb-4593-83f9-5b25f9ccac01 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:45.279787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:45.362140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e163500-5aeb-4593-83f9-5b25f9ccac01 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:45.396026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486403b-4a4c-4a39-83a9-c4df2d505596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:45.396966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486403b-4a4c-4a39-83a9-c4df2d505596 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:45.397242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486403b-4a4c-4a39-83a9-c4df2d505596 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:45.397518 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486403b-4a4c-4a39-83a9-c4df2d505596 None admin] PATH_INFO: `/v3/users/c9e03830e6db45b7a8f767dea625c38e` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:45.412988 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486403b-4a4c-4a39-83a9-c4df2d505596 None admin] RBAC: Authorizing `identity:get_user(user_id=c9e03830e6db45b7a8f767dea625c38e)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:45.414502 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486403b-4a4c-4a39-83a9-c4df2d505596 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:45.416366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 534/1056] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:54:45 2022] GET /identity/v3/users/c9e03830e6db45b7a8f767dea625c38e => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:45.422080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef003b0-f383-47c9-8b3a-95bbdcd78cee None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:45.433630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:45.525719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef003b0-f383-47c9-8b3a-95bbdcd78cee None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:45.558639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c65bf-1695-4dba-ba3a-72be8a58cb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:45.559285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c65bf-1695-4dba-ba3a-72be8a58cb29 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:45.559561 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c65bf-1695-4dba-ba3a-72be8a58cb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:45.559835 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c65bf-1695-4dba-ba3a-72be8a58cb29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:45.629358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c65bf-1695-4dba-ba3a-72be8a58cb29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:45.630741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c65bf-1695-4dba-ba3a-72be8a58cb29 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:45.641552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:45.736798 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 523/1057] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:54:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:54:45.742748 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486403b-4a4c-4a39-83a9-c4df2d505596 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:45.818459 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:45.845170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486403b-4a4c-4a39-83a9-c4df2d505596 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:45.886964 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0226191-b280-4b49-be6d-dad2b4a9881f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:45.887720 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0226191-b280-4b49-be6d-dad2b4a9881f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:45.887999 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0226191-b280-4b49-be6d-dad2b4a9881f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:45.888242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0226191-b280-4b49-be6d-dad2b4a9881f None admin] PATH_INFO: `/v3/projects/6eabec0485e949329f72ca0f37f3d38b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:45.889463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0226191-b280-4b49-be6d-dad2b4a9881f None admin] RBAC: Authorizing `identity:get_project(project_id=6eabec0485e949329f72ca0f37f3d38b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:45.890879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0226191-b280-4b49-be6d-dad2b4a9881f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:45.893138 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 535/1058] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:54:45 2022] GET /identity/v3/projects/6eabec0485e949329f72ca0f37f3d38b => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:45.899280 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c65bf-1695-4dba-ba3a-72be8a58cb29 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:45.997658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c65bf-1695-4dba-ba3a-72be8a58cb29 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:46.007365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:46.033651 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7ed91-ce4b-46b9-a8fe-ccd276ebebcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:46.034308 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7ed91-ce4b-46b9-a8fe-ccd276ebebcd None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:46.034584 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7ed91-ce4b-46b9-a8fe-ccd276ebebcd None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:46.034856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7ed91-ce4b-46b9-a8fe-ccd276ebebcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:46.035714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7ed91-ce4b-46b9-a8fe-ccd276ebebcd None admin] RBAC: Adding query filter params (role.id=b1e13c449fa64b3e88b6832bb767214b, scope.project.id=6eabec0485e949329f72ca0f37f3d38b, user.id=c9e03830e6db45b7a8f767dea625c38e) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:46.036020 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7ed91-ce4b-46b9-a8fe-ccd276ebebcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:46.037661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7ed91-ce4b-46b9-a8fe-ccd276ebebcd None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:46.048444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 524/1059] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:54:45 2022] GET /identity/v3/role_assignments?user.id=c9e03830e6db45b7a8f767dea625c38e&scope.project.id=6eabec0485e949329f72ca0f37f3d38b&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:46.768981 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318c705-d82b-481e-ac38-9d73d42df622 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:46.769675 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318c705-d82b-481e-ac38-9d73d42df622 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:46.769675 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318c705-d82b-481e-ac38-9d73d42df622 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:46.771044 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 536/1060] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:46.776201 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4d7ad-ef3f-4893-82ec-3c6498b4f992 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:46.776726 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4d7ad-ef3f-4893-82ec-3c6498b4f992 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:46.776726 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4d7ad-ef3f-4893-82ec-3c6498b4f992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:46.793628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d6f4d7ad-ef3f-4893-82ec-3c6498b4f992 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:46.794796 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d6f4d7ad-ef3f-4893-82ec-3c6498b4f992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:46.870624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 525/1061] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:46.893903 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318c705-d82b-481e-ac38-9d73d42df622 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:46.970981 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badcf6af-0915-488c-a548-e4d31fed2be2 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:46.971222 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badcf6af-0915-488c-a548-e4d31fed2be2 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:46.971501 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badcf6af-0915-488c-a548-e4d31fed2be2 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:46.972971 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 526/1062] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:54:46.977922 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2668c7b-8482-412f-9720-894241dfde8f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:46.978178 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2668c7b-8482-412f-9720-894241dfde8f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:46.978465 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2668c7b-8482-412f-9720-894241dfde8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:46.996027 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a2668c7b-8482-412f-9720-894241dfde8f None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:46.997264 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a2668c7b-8482-412f-9720-894241dfde8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:46.997485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318c705-d82b-481e-ac38-9d73d42df622 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:47.034751 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f284d6-81a0-4468-8ffd-d1126edf2885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:47.035890 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f284d6-81a0-4468-8ffd-d1126edf2885 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:47.036308 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f284d6-81a0-4468-8ffd-d1126edf2885 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:47.036774 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f284d6-81a0-4468-8ffd-d1126edf2885 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:47.051976 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f284d6-81a0-4468-8ffd-d1126edf2885 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:54:47.065651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-51f284d6-81a0-4468-8ffd-d1126edf2885 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 25 15:54:47.065651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:47.065651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:47.065651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:47.065651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:47.065651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:47.065651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:47.065651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:47.065651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:47.065651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:47.065651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:47.065651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:47.065651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:54:47.065651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:54:47.065651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:54:47.065651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:54:47.066626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:54:47.066626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:54:47.066626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:54:47.066626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:54:47.066626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:47.066626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:47.066626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:47.066626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:47.066626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:47.066626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:47.066626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:47.066626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:47.066626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:47.066626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:47.066626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:47.066626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:47.072954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:47.072954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:47.072954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:47.072954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:47.072954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:54:47.072954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:54:47.072954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:54:47.072954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:54:47.072954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:54:47.072954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:54:47.072954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 25 15:54:47.072954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:47.072954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 537/1063] 10.222.0.13 () {56 vars in 1120 bytes} [Sat Jun 25 15:54:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:47.072954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f284d6-81a0-4468-8ffd-d1126edf2885 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:47.073718 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 527/1064] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:47.077874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:47.088495 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2668c7b-8482-412f-9720-894241dfde8f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:47.122360 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:47.181941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f284d6-81a0-4468-8ffd-d1126edf2885 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:47.196664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2668c7b-8482-412f-9720-894241dfde8f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:47.218432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219310f-8ef2-4bef-9a73-6ced58c29fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:47.219127 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219310f-8ef2-4bef-9a73-6ced58c29fad None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:47.219382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219310f-8ef2-4bef-9a73-6ced58c29fad None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:47.219656 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219310f-8ef2-4bef-9a73-6ced58c29fad None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:47.220433 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c219310f-8ef2-4bef-9a73-6ced58c29fad None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:47.220745 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c219310f-8ef2-4bef-9a73-6ced58c29fad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:47.222111 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c219310f-8ef2-4bef-9a73-6ced58c29fad None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:47.230775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 538/1065] 10.222.0.13 () {56 vars in 1115 bytes} [Sat Jun 25 15:54:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:47.236435 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219310f-8ef2-4bef-9a73-6ced58c29fad None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:47.241647 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc6677-22ec-4a3e-a03d-c5dc159a1bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:47.242462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc6677-22ec-4a3e-a03d-c5dc159a1bbf None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:47.242743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc6677-22ec-4a3e-a03d-c5dc159a1bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:47.243020 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc6677-22ec-4a3e-a03d-c5dc159a1bbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:47.304303 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:47.318188 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc6677-22ec-4a3e-a03d-c5dc159a1bbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:47.319684 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc6677-22ec-4a3e-a03d-c5dc159a1bbf None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:47.342829 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219310f-8ef2-4bef-9a73-6ced58c29fad None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:47.350731 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:47.381656 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a480c-72b0-4e5d-90cc-12088e246731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:47.383077 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a480c-72b0-4e5d-90cc-12088e246731 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:47.383613 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a480c-72b0-4e5d-90cc-12088e246731 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:47.384080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a480c-72b0-4e5d-90cc-12088e246731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:47.399004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a480c-72b0-4e5d-90cc-12088e246731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:47.400842 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a480c-72b0-4e5d-90cc-12088e246731 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:47.420363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-b31a480c-72b0-4e5d-90cc-12088e246731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:47.420363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:47.420363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:47.420363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:47.420363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:47.420363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:47.420363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:47.420363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:47.420363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:47.420363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:47.420363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:47.420363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:47.420363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:47.420363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:47.420363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:47.421483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:47.421483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:47.421483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:47.421483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:47.421483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:47.421483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:47.421483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:47.421483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:47.421483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:47.421483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:47.421483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:47.421483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:47.421483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:47.421483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:47.421483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:47.421483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:47.421483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:47.422540 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:47.422540 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:47.422540 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:47.422540 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:47.422540 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 539/1066] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:47.426733 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 528/1067] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:54:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:54:47.427196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a480c-72b0-4e5d-90cc-12088e246731 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:47.433365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc6677-22ec-4a3e-a03d-c5dc159a1bbf None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:47.534575 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a480c-72b0-4e5d-90cc-12088e246731 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:47.534994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc6677-22ec-4a3e-a03d-c5dc159a1bbf None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:47.552454 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:47.564914 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:47.571905 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea88e39-8c92-4e3b-bd72-3e74dffa41ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:47.573424 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea88e39-8c92-4e3b-bd72-3e74dffa41ad None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:47.573424 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea88e39-8c92-4e3b-bd72-3e74dffa41ad None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:47.573424 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea88e39-8c92-4e3b-bd72-3e74dffa41ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:47.574657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8082a-9d5a-4fd0-b3b5-04d01bea0428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:47.575326 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8082a-9d5a-4fd0-b3b5-04d01bea0428 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:47.575608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8082a-9d5a-4fd0-b3b5-04d01bea0428 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:47.575880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8082a-9d5a-4fd0-b3b5-04d01bea0428 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:47.576631 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8082a-9d5a-4fd0-b3b5-04d01bea0428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:47.576941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8082a-9d5a-4fd0-b3b5-04d01bea0428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:47.578312 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8082a-9d5a-4fd0-b3b5-04d01bea0428 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:47.585865 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea88e39-8c92-4e3b-bd72-3e74dffa41ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:47.588548 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea88e39-8c92-4e3b-bd72-3e74dffa41ad None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:47.596045 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 540/1068] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:47.597101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8082a-9d5a-4fd0-b3b5-04d01bea0428 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:47.612488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-9ea88e39-8c92-4e3b-bd72-3e74dffa41ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:47.612488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:47.612488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:47.612488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:47.612488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:47.612488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:47.612488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:47.612488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:47.612488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:47.612488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:47.612488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:47.612488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:47.612488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:47.612488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:47.612488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:47.613442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:47.613442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:47.613442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:47.613442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:47.613442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:47.613442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:47.613442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:47.613442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:47.613442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:47.613442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:47.613442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:47.613442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:47.613442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:47.613442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:47.613442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:47.613442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:47.613442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:47.614354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:47.614354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:47.614354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:47.614354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:47.614354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 529/1069] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:47.619863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea88e39-8c92-4e3b-bd72-3e74dffa41ad None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:47.710370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8082a-9d5a-4fd0-b3b5-04d01bea0428 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:47.730921 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea88e39-8c92-4e3b-bd72-3e74dffa41ad None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:47.747673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c11db5-0d79-4962-a1d7-5ed30f189474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:47.748503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c11db5-0d79-4962-a1d7-5ed30f189474 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:47.748797 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c11db5-0d79-4962-a1d7-5ed30f189474 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:47.749056 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c11db5-0d79-4962-a1d7-5ed30f189474 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:47.756748 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c11db5-0d79-4962-a1d7-5ed30f189474 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:47.758522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c11db5-0d79-4962-a1d7-5ed30f189474 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:47.767725 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06205b28-6b17-4f37-b376-d943f2355c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:47.768395 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06205b28-6b17-4f37-b376-d943f2355c17 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:47.768705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06205b28-6b17-4f37-b376-d943f2355c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:47.768980 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06205b28-6b17-4f37-b376-d943f2355c17 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:47.769715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06205b28-6b17-4f37-b376-d943f2355c17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:47.770023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06205b28-6b17-4f37-b376-d943f2355c17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:47.771404 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06205b28-6b17-4f37-b376-d943f2355c17 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:47.773205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-81c11db5-0d79-4962-a1d7-5ed30f189474 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 25 15:54:47.773205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:47.773205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:47.773205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:47.773205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:47.773205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:47.773205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:47.773205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:47.773205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:47.773205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:47.773205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:47.773205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:47.773205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:54:47.773205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:54:47.773205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:54:47.773205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:54:47.774442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:47.774442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:47.774442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:54:47.774442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:47.774442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:54:47.774442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:47.774442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:47.774442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:47.774442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:47.774442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:47.774442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:47.774442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:47.774442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:47.774442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:47.774442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:47.774442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:47.774442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:47.774442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:47.775486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:47.775486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:47.775486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:54:47.775486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:54:47.775486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:54:47.775486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:54:47.775486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:54:47.775486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:54:47.775486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 25 15:54:47.775486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:47.775486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 541/1070] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Jun 25 15:54:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:47.780217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c11db5-0d79-4962-a1d7-5ed30f189474 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:47.787130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 530/1071] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:47.833971 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:47.909260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c11db5-0d79-4962-a1d7-5ed30f189474 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:47.947768 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230b416-abc4-4c7f-9a42-aa54866f9347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:47.948607 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230b416-abc4-4c7f-9a42-aa54866f9347 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:47.948864 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230b416-abc4-4c7f-9a42-aa54866f9347 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:47.949765 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230b416-abc4-4c7f-9a42-aa54866f9347 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:47.950032 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230b416-abc4-4c7f-9a42-aa54866f9347 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:47.950346 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230b416-abc4-4c7f-9a42-aa54866f9347 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:47.952099 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230b416-abc4-4c7f-9a42-aa54866f9347 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:47.971102 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 542/1072] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Jun 25 15:54:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:47.977371 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06205b28-6b17-4f37-b376-d943f2355c17 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:48.011968 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929eb103-ded9-48d8-b24d-37a7997cde44 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:48.012250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929eb103-ded9-48d8-b24d-37a7997cde44 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:48.012587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929eb103-ded9-48d8-b24d-37a7997cde44 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:48.013968 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 543/1073] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:54:48.019370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95029f84-f1d4-4588-b7d6-49527a4537f4 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:48.019664 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95029f84-f1d4-4588-b7d6-49527a4537f4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:48.019942 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95029f84-f1d4-4588-b7d6-49527a4537f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:48.033357 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:48.036728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-95029f84-f1d4-4588-b7d6-49527a4537f4 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:48.037942 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-95029f84-f1d4-4588-b7d6-49527a4537f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:48.094042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06205b28-6b17-4f37-b376-d943f2355c17 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:48.110719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 544/1074] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:48.128607 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5355eca-09b6-4b75-8d7c-c0ac0626604d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:48.129394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5355eca-09b6-4b75-8d7c-c0ac0626604d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:48.129674 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5355eca-09b6-4b75-8d7c-c0ac0626604d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:48.129943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5355eca-09b6-4b75-8d7c-c0ac0626604d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:48.133103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95029f84-f1d4-4588-b7d6-49527a4537f4 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:48.140660 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5355eca-09b6-4b75-8d7c-c0ac0626604d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:48.142435 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5355eca-09b6-4b75-8d7c-c0ac0626604d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:48.159317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-e5355eca-09b6-4b75-8d7c-c0ac0626604d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:48.159317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:48.159317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:48.159317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:48.159317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:48.159317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:48.159317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:48.159317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:48.159317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:48.159317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:48.159317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:48.159317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:48.159317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:48.159317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:48.159317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:48.160277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:48.160277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:48.160277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:48.160277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:48.160277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:48.160277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:48.160277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:48.160277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:48.160277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:48.160277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:48.160277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:48.160277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:48.160277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:48.160277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:48.160277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:48.160277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:48.160277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:48.161658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:48.161658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:48.161658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:48.161658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:48.161658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 531/1075] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:48.161975 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:48.166496 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5355eca-09b6-4b75-8d7c-c0ac0626604d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:48.233826 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95029f84-f1d4-4588-b7d6-49527a4537f4 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:48.259463 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:48.265830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5355eca-09b6-4b75-8d7c-c0ac0626604d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:48.268088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15571981-65db-4314-8807-e8e5324d827e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:48.268923 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15571981-65db-4314-8807-e8e5324d827e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:48.269231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15571981-65db-4314-8807-e8e5324d827e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:48.269501 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15571981-65db-4314-8807-e8e5324d827e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:48.279898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15571981-65db-4314-8807-e8e5324d827e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:54:48.287542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-15571981-65db-4314-8807-e8e5324d827e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:54:48.287542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:48.287542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:48.287542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:48.287542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:48.287542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:48.287542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:48.287542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:48.287542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:48.287542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:48.287542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:48.287542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:48.287542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:54:48.287542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:54:48.287542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:54:48.287542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:54:48.289083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:54:48.289083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:54:48.289083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:54:48.289083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:54:48.289083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:48.289083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:48.289083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:48.289083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:48.289083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:48.289083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:48.289083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:48.289083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:48.289083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:48.289083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:48.289083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:48.289083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:48.289951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:48.289951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:48.289951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:48.289951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:48.289951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:54:48.289951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:54:48.289951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:54:48.289951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:54:48.289951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:54:48.289951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:54:48.289951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:54:48.289951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:48.289951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 545/1076] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:54:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:48.295363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15571981-65db-4314-8807-e8e5324d827e None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:48.301319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ad56f-a40f-43ce-a8d3-c73f74d6e5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:48.301988 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ad56f-a40f-43ce-a8d3-c73f74d6e5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:48.302269 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ad56f-a40f-43ce-a8d3-c73f74d6e5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:48.302545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ad56f-a40f-43ce-a8d3-c73f74d6e5d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:48.303274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ad56f-a40f-43ce-a8d3-c73f74d6e5d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:48.303582 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ad56f-a40f-43ce-a8d3-c73f74d6e5d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:48.305654 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ad56f-a40f-43ce-a8d3-c73f74d6e5d2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:48.318211 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 532/1077] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:48.324163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ad56f-a40f-43ce-a8d3-c73f74d6e5d2 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:48.378657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:48.395347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15571981-65db-4314-8807-e8e5324d827e None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:48.421282 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ad56f-a40f-43ce-a8d3-c73f74d6e5d2 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:48.429727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a9de6-5ad1-465c-80b1-55d5f0dfda36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:48.430396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a9de6-5ad1-465c-80b1-55d5f0dfda36 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:48.430678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a9de6-5ad1-465c-80b1-55d5f0dfda36 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:48.430952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a9de6-5ad1-465c-80b1-55d5f0dfda36 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:48.431693 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a9de6-5ad1-465c-80b1-55d5f0dfda36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:48.432008 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a9de6-5ad1-465c-80b1-55d5f0dfda36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:48.433392 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a9de6-5ad1-465c-80b1-55d5f0dfda36 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:48.441458 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 546/1078] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:54:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:48.447525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a9de6-5ad1-465c-80b1-55d5f0dfda36 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:48.455620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f0055-f464-46b9-a1fe-807af89fcea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:48.456439 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f0055-f464-46b9-a1fe-807af89fcea5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:48.456730 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f0055-f464-46b9-a1fe-807af89fcea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:48.457000 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f0055-f464-46b9-a1fe-807af89fcea5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:48.464556 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f0055-f464-46b9-a1fe-807af89fcea5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:48.466004 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f0055-f464-46b9-a1fe-807af89fcea5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:48.479624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-321f0055-f464-46b9-a1fe-807af89fcea5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:48.479624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:48.479624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:48.479624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:48.479624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:48.479624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:48.479624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:48.479624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:48.479624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:48.479624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:48.479624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:48.479624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:48.479624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:54:48.479624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:54:48.479624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:54:48.480711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:54:48.480711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:48.480711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:48.480711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:48.480711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:48.480711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:48.480711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:48.480711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:48.480711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:48.480711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:48.480711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:48.480711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:48.480711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:48.480711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:48.480711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:48.480711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:48.480711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:48.481818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:54:48.481818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:54:48.481818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:54:48.481818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:54:48.481818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:54:48.481818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:54:48.481818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:48.481818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:48.481818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 533/1079] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:48.486599 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f0055-f464-46b9-a1fe-807af89fcea5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:48.505670 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:48.551729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a9de6-5ad1-465c-80b1-55d5f0dfda36 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:48.587050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a22f7-eda4-4389-9f58-4ccad8574b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:48.587807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f0055-f464-46b9-a1fe-807af89fcea5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:48.587969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a22f7-eda4-4389-9f58-4ccad8574b27 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:48.588148 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a22f7-eda4-4389-9f58-4ccad8574b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:48.588479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a22f7-eda4-4389-9f58-4ccad8574b27 None admin] PATH_INFO: `/v3/users/c9e03830e6db45b7a8f767dea625c38e` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:48.589820 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a22f7-eda4-4389-9f58-4ccad8574b27 None admin] RBAC: Authorizing `identity:get_user(user_id=c9e03830e6db45b7a8f767dea625c38e)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:48.591272 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931a22f7-eda4-4389-9f58-4ccad8574b27 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:48.592960 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 547/1080] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:54:48 2022] GET /identity/v3/users/c9e03830e6db45b7a8f767dea625c38e => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:48.598802 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a22f7-eda4-4389-9f58-4ccad8574b27 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:48.616931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:48.622521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacaeec2-534e-4a7e-b14e-2f6dcfdf8155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:48.623187 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaeec2-534e-4a7e-b14e-2f6dcfdf8155 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:48.623466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaeec2-534e-4a7e-b14e-2f6dcfdf8155 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:48.623740 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaeec2-534e-4a7e-b14e-2f6dcfdf8155 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:48.624556 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacaeec2-534e-4a7e-b14e-2f6dcfdf8155 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:48.624864 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacaeec2-534e-4a7e-b14e-2f6dcfdf8155 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:48.626300 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacaeec2-534e-4a7e-b14e-2f6dcfdf8155 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:48.638344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 534/1081] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:54:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:48.644550 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacaeec2-534e-4a7e-b14e-2f6dcfdf8155 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:48.698227 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a22f7-eda4-4389-9f58-4ccad8574b27 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:48.720599 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:48.739021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1bd215-27f4-4b65-9a10-785b543348b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:48.739884 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1bd215-27f4-4b65-9a10-785b543348b6 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:48.740166 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1bd215-27f4-4b65-9a10-785b543348b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:48.740464 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1bd215-27f4-4b65-9a10-785b543348b6 None admin] PATH_INFO: `/v3/projects/6eabec0485e949329f72ca0f37f3d38b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:48.741892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1bd215-27f4-4b65-9a10-785b543348b6 None admin] RBAC: Authorizing `identity:get_project(project_id=6eabec0485e949329f72ca0f37f3d38b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:48.742874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacaeec2-534e-4a7e-b14e-2f6dcfdf8155 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:48.743482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1bd215-27f4-4b65-9a10-785b543348b6 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:48.745256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 548/1082] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:54:48 2022] GET /identity/v3/projects/6eabec0485e949329f72ca0f37f3d38b => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:48.751048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1bd215-27f4-4b65-9a10-785b543348b6 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:48.780986 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1416f-19c2-4fe8-acd2-e3a8eef573f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:48.781875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1416f-19c2-4fe8-acd2-e3a8eef573f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:48.782147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1416f-19c2-4fe8-acd2-e3a8eef573f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:48.782403 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1416f-19c2-4fe8-acd2-e3a8eef573f2 None admin] PATH_INFO: `/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/3622c5ccb7534e7c8e8342b4f6643bc3/roles/c38990a9778e4b2abd3376f9e066d3a5` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:48.784707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f1416f-19c2-4fe8-acd2-e3a8eef573f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3600648eb94b434b8c797d2f454e46b8, user_id=3622c5ccb7534e7c8e8342b4f6643bc3, role_id=c38990a9778e4b2abd3376f9e066d3a5)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:48.786138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f1416f-19c2-4fe8-acd2-e3a8eef573f2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:48.799718 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 535/1083] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:54:48 2022] PUT /identity/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/3622c5ccb7534e7c8e8342b4f6643bc3/roles/c38990a9778e4b2abd3376f9e066d3a5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 15:54:48.843597 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:48.862187 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1bd215-27f4-4b65-9a10-785b543348b6 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:48.899039 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf901d-8295-4eb8-8a8e-5acfe0803473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:48.899848 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf901d-8295-4eb8-8a8e-5acfe0803473 None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:48.900124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf901d-8295-4eb8-8a8e-5acfe0803473 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:48.900397 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf901d-8295-4eb8-8a8e-5acfe0803473 None admin] PATH_INFO: `/v3/projects/6eabec0485e949329f72ca0f37f3d38b/users/c9e03830e6db45b7a8f767dea625c38e/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:48.902468 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf901d-8295-4eb8-8a8e-5acfe0803473 None admin] RBAC: Authorizing `identity:create_grant(project_id=6eabec0485e949329f72ca0f37f3d38b, user_id=c9e03830e6db45b7a8f767dea625c38e, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:48.903906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf901d-8295-4eb8-8a8e-5acfe0803473 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:48.919100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 549/1084] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:54:48 2022] PUT /identity/v3/projects/6eabec0485e949329f72ca0f37f3d38b/users/c9e03830e6db45b7a8f767dea625c38e/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 15:54:50.815055 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774f1ae-45a1-4d2b-91e2-791f0cca1179 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:50.815633 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774f1ae-45a1-4d2b-91e2-791f0cca1179 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:50.815633 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774f1ae-45a1-4d2b-91e2-791f0cca1179 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:50.816992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 536/1085] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:50.822380 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573f29b-b743-4c81-b1a4-6fc2dfdae291 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:50.822835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573f29b-b743-4c81-b1a4-6fc2dfdae291 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:50.822927 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573f29b-b743-4c81-b1a4-6fc2dfdae291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:50.840484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7573f29b-b743-4c81-b1a4-6fc2dfdae291 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:50.841697 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7573f29b-b743-4c81-b1a4-6fc2dfdae291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:50.869797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3fe77-2850-47cf-9b4a-b2281cd521a0 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:50.870084 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3fe77-2850-47cf-9b4a-b2281cd521a0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:50.870353 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3fe77-2850-47cf-9b4a-b2281cd521a0 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:50.871853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 537/1086] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:50.876336 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e939c-3705-4e1f-9577-6dcfcb8de26c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:50.876639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e939c-3705-4e1f-9577-6dcfcb8de26c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:50.876929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e939c-3705-4e1f-9577-6dcfcb8de26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:50.893351 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-bd2e939c-3705-4e1f-9577-6dcfcb8de26c None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:50.894527 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bd2e939c-3705-4e1f-9577-6dcfcb8de26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:50.914588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 550/1087] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:50.937600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573f29b-b743-4c81-b1a4-6fc2dfdae291 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:50.960646 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 538/1088] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:54:50.983577 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e939c-3705-4e1f-9577-6dcfcb8de26c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:51.010576 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:51.038408 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573f29b-b743-4c81-b1a4-6fc2dfdae291 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:51.049057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:51.073683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd1f90-e26d-415e-8f2c-946ef053be09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:51.074526 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd1f90-e26d-415e-8f2c-946ef053be09 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:51.074803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd1f90-e26d-415e-8f2c-946ef053be09 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:51.075075 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd1f90-e26d-415e-8f2c-946ef053be09 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:51.083688 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e939c-3705-4e1f-9577-6dcfcb8de26c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:51.084781 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd1f90-e26d-415e-8f2c-946ef053be09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:54:51.093018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-4afd1f90-e26d-415e-8f2c-946ef053be09 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 25 15:54:51.093018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:51.093018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:51.093018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:51.093018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:51.093018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:51.093018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:51.093018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:51.093018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:51.093018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:51.093018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:51.093018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:51.093018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:54:51.093018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:54:51.093018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:54:51.093018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:54:51.093989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:54:51.093989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:54:51.093989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:54:51.093989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:54:51.093989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:51.093989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:51.093989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:51.093989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:51.093989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:51.093989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:51.093989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:51.093989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:51.093989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:51.093989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:51.093989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:51.093989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:51.095608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:51.095608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:51.095608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:51.095608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:51.095608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:54:51.095608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:54:51.095608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:54:51.095608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:54:51.095608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:54:51.095608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:54:51.095608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 25 15:54:51.095608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:51.095608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 551/1089] 10.222.0.13 () {56 vars in 1120 bytes} [Sat Jun 25 15:54:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:51.100122 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd1f90-e26d-415e-8f2c-946ef053be09 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:51.121435 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b36f6-83f5-480f-9cba-cd864a0adc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:51.122249 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b36f6-83f5-480f-9cba-cd864a0adc11 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:51.122499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b36f6-83f5-480f-9cba-cd864a0adc11 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:51.122764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b36f6-83f5-480f-9cba-cd864a0adc11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:51.133176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4b36f6-83f5-480f-9cba-cd864a0adc11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:54:51.140794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-2d4b36f6-83f5-480f-9cba-cd864a0adc11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:54:51.140794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:51.140794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:51.140794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:51.140794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:51.140794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:51.140794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:51.140794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:51.140794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:51.140794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:51.140794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:51.140794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:51.140794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:54:51.140794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:54:51.140794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:54:51.140794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:54:51.142057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:54:51.142057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:54:51.142057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:54:51.142057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:54:51.142057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:51.142057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:51.142057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:51.142057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:51.142057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:51.142057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:51.142057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:51.142057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:51.142057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:51.142057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:51.142057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:51.142057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:51.142929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:51.142929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:51.142929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:51.142929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:51.142929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:54:51.142929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:54:51.142929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:54:51.142929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:54:51.142929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:54:51.142929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:54:51.142929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:54:51.142929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:51.142929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 539/1090] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:54:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:51.148660 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b36f6-83f5-480f-9cba-cd864a0adc11 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:51.202327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd1f90-e26d-415e-8f2c-946ef053be09 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:51.235285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2097d-5fa8-4b21-98d6-a2846e3dd787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:51.235959 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2097d-5fa8-4b21-98d6-a2846e3dd787 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:51.236241 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2097d-5fa8-4b21-98d6-a2846e3dd787 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:51.236642 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2097d-5fa8-4b21-98d6-a2846e3dd787 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:51.237414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2097d-5fa8-4b21-98d6-a2846e3dd787 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:51.237723 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2097d-5fa8-4b21-98d6-a2846e3dd787 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:51.239087 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2097d-5fa8-4b21-98d6-a2846e3dd787 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:51.240673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:51.246550 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b36f6-83f5-480f-9cba-cd864a0adc11 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:51.248287 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 552/1091] 10.222.0.13 () {56 vars in 1115 bytes} [Sat Jun 25 15:54:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:51.253840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2097d-5fa8-4b21-98d6-a2846e3dd787 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:51.277734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:51.287309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d927286-b04a-4f99-9281-61a58b6efae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:51.288046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d927286-b04a-4f99-9281-61a58b6efae2 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:51.288295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d927286-b04a-4f99-9281-61a58b6efae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:51.288772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d927286-b04a-4f99-9281-61a58b6efae2 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:51.289530 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d927286-b04a-4f99-9281-61a58b6efae2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:51.289831 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d927286-b04a-4f99-9281-61a58b6efae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:51.291207 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d927286-b04a-4f99-9281-61a58b6efae2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:51.299492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 540/1092] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:54:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:51.305300 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d927286-b04a-4f99-9281-61a58b6efae2 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:51.359766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2097d-5fa8-4b21-98d6-a2846e3dd787 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:51.396511 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309c535-c20c-49d4-86e4-ee109542b5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:51.397305 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309c535-c20c-49d4-86e4-ee109542b5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:51.397588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309c535-c20c-49d4-86e4-ee109542b5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:51.398395 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309c535-c20c-49d4-86e4-ee109542b5e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:51.406248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d927286-b04a-4f99-9281-61a58b6efae2 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:51.408887 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309c535-c20c-49d4-86e4-ee109542b5e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:51.410328 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309c535-c20c-49d4-86e4-ee109542b5e3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:51.427579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-2309c535-c20c-49d4-86e4-ee109542b5e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:51.427579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:51.427579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:51.427579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:51.427579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:51.427579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:51.427579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:51.427579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:51.427579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:51.427579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:51.427579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:51.427579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:51.427579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:51.427579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:51.427579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:51.428563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:51.428563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:51.428563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:51.428563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:51.428563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:51.428563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:51.428563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:51.428563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:51.428563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:51.428563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:51.428563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:51.428563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:51.428563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:51.428563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:51.428563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:51.428563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:51.428563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:51.429571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:51.429571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:51.429571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:51.429571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:51.429571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 553/1093] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:54:51.435046 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309c535-c20c-49d4-86e4-ee109542b5e3 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:51.439555 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79835825-1349-41e3-a79c-9ea5bab9265d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:51.440440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79835825-1349-41e3-a79c-9ea5bab9265d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:51.440733 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79835825-1349-41e3-a79c-9ea5bab9265d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:51.441005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79835825-1349-41e3-a79c-9ea5bab9265d None admin] PATH_INFO: `/v3/users/c9e03830e6db45b7a8f767dea625c38e` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:51.442213 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79835825-1349-41e3-a79c-9ea5bab9265d None admin] RBAC: Authorizing `identity:get_user(user_id=c9e03830e6db45b7a8f767dea625c38e)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:51.443648 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79835825-1349-41e3-a79c-9ea5bab9265d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:51.445645 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 541/1094] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:54:51 2022] GET /identity/v3/users/c9e03830e6db45b7a8f767dea625c38e => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:51.451157 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79835825-1349-41e3-a79c-9ea5bab9265d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:51.487168 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:51.510529 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:51.542614 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309c535-c20c-49d4-86e4-ee109542b5e3 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:51.554930 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79835825-1349-41e3-a79c-9ea5bab9265d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:51.576265 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4ee92-4737-4bee-890c-efccdd1aa19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:51.576954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4ee92-4737-4bee-890c-efccdd1aa19c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:51.577263 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4ee92-4737-4bee-890c-efccdd1aa19c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:51.577521 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4ee92-4737-4bee-890c-efccdd1aa19c None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:51.578229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4ee92-4737-4bee-890c-efccdd1aa19c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:51.578533 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4ee92-4737-4bee-890c-efccdd1aa19c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:51.579874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4ee92-4737-4bee-890c-efccdd1aa19c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:51.589061 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b9b23-67e0-40c7-931f-1c51831c22f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:51.589723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b9b23-67e0-40c7-931f-1c51831c22f0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:51.590050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b9b23-67e0-40c7-931f-1c51831c22f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:51.590283 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b9b23-67e0-40c7-931f-1c51831c22f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:51.592384 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 554/1095] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:51.598049 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4ee92-4737-4bee-890c-efccdd1aa19c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:51.661485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b9b23-67e0-40c7-931f-1c51831c22f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:51.662854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b9b23-67e0-40c7-931f-1c51831c22f0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:51.699532 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4ee92-4737-4bee-890c-efccdd1aa19c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:51.708578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:51.722062 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:51.735447 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f554377-f986-4ab8-abb8-3357b5c1ea80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:51.736246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f554377-f986-4ab8-abb8-3357b5c1ea80 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:51.736561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f554377-f986-4ab8-abb8-3357b5c1ea80 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:51.736837 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f554377-f986-4ab8-abb8-3357b5c1ea80 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:51.745047 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f554377-f986-4ab8-abb8-3357b5c1ea80 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:51.746496 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f554377-f986-4ab8-abb8-3357b5c1ea80 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:51.761186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-1f554377-f986-4ab8-abb8-3357b5c1ea80 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 25 15:54:51.761186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:51.761186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:51.761186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:51.761186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:51.761186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:51.761186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:51.761186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:51.761186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:51.761186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:51.761186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:51.761186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:51.761186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:54:51.761186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:54:51.761186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:54:51.761186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:54:51.762268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:51.762268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:51.762268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:54:51.762268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:51.762268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:54:51.762268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:54:51.762268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:51.762268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:51.762268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:51.762268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:51.762268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:51.762268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:51.762268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:51.762268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:51.762268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:51.762268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:51.762268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:51.762268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:51.763217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:51.763217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:51.763217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:54:51.763217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:54:51.763217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:54:51.763217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:54:51.763217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:54:51.763217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:54:51.763217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 25 15:54:51.763217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:54:51.763217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 555/1096] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Jun 25 15:54:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:51.768439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f554377-f986-4ab8-abb8-3357b5c1ea80 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:51.771656 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 542/1097] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:54:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 25 15:54:51.777087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b9b23-67e0-40c7-931f-1c51831c22f0 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:51.874199 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f554377-f986-4ab8-abb8-3357b5c1ea80 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:51.877194 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b9b23-67e0-40c7-931f-1c51831c22f0 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:51.907219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36b69d-6c30-4bfc-a103-736b16c05efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:51.907889 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36b69d-6c30-4bfc-a103-736b16c05efd None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:51.908172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36b69d-6c30-4bfc-a103-736b16c05efd None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:51.908525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36b69d-6c30-4bfc-a103-736b16c05efd None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:51.909312 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36b69d-6c30-4bfc-a103-736b16c05efd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:51.909621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36b69d-6c30-4bfc-a103-736b16c05efd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:51.911050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36b69d-6c30-4bfc-a103-736b16c05efd None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:51.911405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48d981-4d20-4fb6-9a2d-230946970467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:51.912320 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48d981-4d20-4fb6-9a2d-230946970467 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:51.912575 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48d981-4d20-4fb6-9a2d-230946970467 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:51.912839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48d981-4d20-4fb6-9a2d-230946970467 None admin] PATH_INFO: `/v3/projects/6eabec0485e949329f72ca0f37f3d38b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:51.914132 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48d981-4d20-4fb6-9a2d-230946970467 None admin] RBAC: Authorizing `identity:get_project(project_id=6eabec0485e949329f72ca0f37f3d38b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:51.915594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48d981-4d20-4fb6-9a2d-230946970467 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:51.917351 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 543/1098] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:54:51 2022] GET /identity/v3/projects/6eabec0485e949329f72ca0f37f3d38b => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:54:51.923283 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48d981-4d20-4fb6-9a2d-230946970467 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:51.931348 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 556/1099] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Jun 25 15:54:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:51.937223 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36b69d-6c30-4bfc-a103-736b16c05efd None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:51.948457 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:51.965907 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:52.022371 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48d981-4d20-4fb6-9a2d-230946970467 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:52.036454 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36b69d-6c30-4bfc-a103-736b16c05efd None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:52.056682 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a4565-332e-4e23-b6e0-b883e0b21f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:52.057331 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a4565-332e-4e23-b6e0-b883e0b21f78 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:52.057611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a4565-332e-4e23-b6e0-b883e0b21f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:52.057880 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a4565-332e-4e23-b6e0-b883e0b21f78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:52.058711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a4565-332e-4e23-b6e0-b883e0b21f78 None admin] RBAC: Adding query filter params (role.id=b1e13c449fa64b3e88b6832bb767214b, scope.project.id=6eabec0485e949329f72ca0f37f3d38b, user.id=c9e03830e6db45b7a8f767dea625c38e) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:52.059017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a4565-332e-4e23-b6e0-b883e0b21f78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:52.060468 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a4565-332e-4e23-b6e0-b883e0b21f78 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:52.066367 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 544/1100] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:54:51 2022] GET /identity/v3/role_assignments?user.id=c9e03830e6db45b7a8f767dea625c38e&scope.project.id=6eabec0485e949329f72ca0f37f3d38b&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:52.070795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75a1f6-ce3b-49ae-b890-ac0dc7670054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:52.071452 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75a1f6-ce3b-49ae-b890-ac0dc7670054 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:52.071732 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75a1f6-ce3b-49ae-b890-ac0dc7670054 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:52.072031 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75a1f6-ce3b-49ae-b890-ac0dc7670054 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:52.143126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75a1f6-ce3b-49ae-b890-ac0dc7670054 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:52.144615 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75a1f6-ce3b-49ae-b890-ac0dc7670054 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:52.173110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:52.250646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 557/1101] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:54:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:54:52.256745 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a4565-332e-4e23-b6e0-b883e0b21f78 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:52.351143 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:52.364557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a4565-332e-4e23-b6e0-b883e0b21f78 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:52.400793 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c2162-6439-49c6-affb-76dc33ca5910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:52.401577 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c2162-6439-49c6-affb-76dc33ca5910 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:52.401865 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c2162-6439-49c6-affb-76dc33ca5910 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:52.402137 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c2162-6439-49c6-affb-76dc33ca5910 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:52.412528 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c2162-6439-49c6-affb-76dc33ca5910 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:52.414063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c2162-6439-49c6-affb-76dc33ca5910 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:52.431472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-926c2162-6439-49c6-affb-76dc33ca5910 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:52.431472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:52.431472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:52.431472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:52.431472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:52.431472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:52.431472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:52.431472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:52.431472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:52.431472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:52.431472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:52.431472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:52.431472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:54:52.431472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:54:52.431472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:54:52.432402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:54:52.432402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:52.432402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:52.432402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:52.432402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:52.432402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:52.432402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:52.432402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:52.432402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:52.432402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:52.432402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:52.432402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:52.432402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:52.432402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:52.432402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:52.432402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:52.432402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:52.433377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:54:52.433377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:54:52.433377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:54:52.433377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:52.433377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 545/1102] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:52.438503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75a1f6-ce3b-49ae-b890-ac0dc7670054 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:52.541703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75a1f6-ce3b-49ae-b890-ac0dc7670054 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:52.568090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:52.577939 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c79056-0808-4cc5-82f8-d2fdd8b932a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:52.578593 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c79056-0808-4cc5-82f8-d2fdd8b932a5 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:52.578867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c79056-0808-4cc5-82f8-d2fdd8b932a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:52.579136 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c79056-0808-4cc5-82f8-d2fdd8b932a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:52.579851 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c79056-0808-4cc5-82f8-d2fdd8b932a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:52.580152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c79056-0808-4cc5-82f8-d2fdd8b932a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:52.581537 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c79056-0808-4cc5-82f8-d2fdd8b932a5 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:52.592900 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 558/1103] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:52.598582 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c2162-6439-49c6-affb-76dc33ca5910 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:52.702016 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c2162-6439-49c6-affb-76dc33ca5910 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:52.735853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97c12b-e4bd-44c3-8dbb-0478918248f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:52.736689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97c12b-e4bd-44c3-8dbb-0478918248f4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:52.736969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97c12b-e4bd-44c3-8dbb-0478918248f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:52.737239 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97c12b-e4bd-44c3-8dbb-0478918248f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:52.744762 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97c12b-e4bd-44c3-8dbb-0478918248f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:52.746232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a97c12b-e4bd-44c3-8dbb-0478918248f4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:52.749184 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:52.761744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-5a97c12b-e4bd-44c3-8dbb-0478918248f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:52.761744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:54:52.761744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:54:52.761744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:54:52.761744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:54:52.761744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:54:52.761744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:54:52.761744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:54:52.761744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:54:52.761744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:54:52.761744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:54:52.761744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:54:52.761744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:54:52.761744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:54:52.761744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:54:52.762763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:54:52.762763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:54:52.762763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:54:52.762763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:54:52.762763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:54:52.762763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:54:52.762763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:54:52.762763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:54:52.762763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:54:52.762763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:54:52.762763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:54:52.762763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:54:52.762763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:54:52.762763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:54:52.762763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:54:52.762763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:54:52.762763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:54:52.764052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:54:52.764052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:54:52.764052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:54:52.764052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:54:52.764052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:54:52.764052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:54:52.764052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:54:52.764052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:54:52.764052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 546/1104] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:54:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:54:52.769261 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c79056-0808-4cc5-82f8-d2fdd8b932a5 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:52.873394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c79056-0808-4cc5-82f8-d2fdd8b932a5 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:52.906006 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905b736-06cb-40d0-a57f-edc29dea740a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:52.906653 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905b736-06cb-40d0-a57f-edc29dea740a None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:52.906930 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905b736-06cb-40d0-a57f-edc29dea740a None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:52.907199 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905b736-06cb-40d0-a57f-edc29dea740a None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:52.907978 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905b736-06cb-40d0-a57f-edc29dea740a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:52.908280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905b736-06cb-40d0-a57f-edc29dea740a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:52.909720 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905b736-06cb-40d0-a57f-edc29dea740a None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:52.921650 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 559/1105] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:54:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:52.927596 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97c12b-e4bd-44c3-8dbb-0478918248f4 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:53.030489 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97c12b-e4bd-44c3-8dbb-0478918248f4 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:53.065511 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38fd9e-9b45-4691-a297-3ef59c873142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:53.066160 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38fd9e-9b45-4691-a297-3ef59c873142 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:53.066439 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38fd9e-9b45-4691-a297-3ef59c873142 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:53.066715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38fd9e-9b45-4691-a297-3ef59c873142 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:53.067550 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d38fd9e-9b45-4691-a297-3ef59c873142 None admin] RBAC: Adding query filter params (role.id=c38990a9778e4b2abd3376f9e066d3a5, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=3622c5ccb7534e7c8e8342b4f6643bc3) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:53.067860 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d38fd9e-9b45-4691-a297-3ef59c873142 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:53.069335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d38fd9e-9b45-4691-a297-3ef59c873142 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:53.075585 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 547/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:54:52 2022] GET /identity/v3/role_assignments?user.id=3622c5ccb7534e7c8e8342b4f6643bc3&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=c38990a9778e4b2abd3376f9e066d3a5 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:54.108675 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ff4df-fa87-4551-a867-2e63181cda1c None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:54.109498 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ff4df-fa87-4551-a867-2e63181cda1c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:54.109632 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ff4df-fa87-4551-a867-2e63181cda1c None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:54.111845 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 560/1107] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:54.120112 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca705bbc-a542-4413-8a24-ca578b9a1d88 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:54.120842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca705bbc-a542-4413-8a24-ca578b9a1d88 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:54.120965 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca705bbc-a542-4413-8a24-ca578b9a1d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:54.148875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ca705bbc-a542-4413-8a24-ca578b9a1d88 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:54.150068 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ca705bbc-a542-4413-8a24-ca578b9a1d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:54.214029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:54.224369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 548/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:54.249803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ff4df-fa87-4551-a867-2e63181cda1c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:54.301725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:54.359800 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ff4df-fa87-4551-a867-2e63181cda1c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:54.395483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e0e33-e0c0-492a-a12a-886d4933ac6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:54.396182 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e0e33-e0c0-492a-a12a-886d4933ac6b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:54.396442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e0e33-e0c0-492a-a12a-886d4933ac6b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:54.396749 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e0e33-e0c0-492a-a12a-886d4933ac6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:54.398002 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e0e33-e0c0-492a-a12a-886d4933ac6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:54.399419 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e0e33-e0c0-492a-a12a-886d4933ac6b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:54.401068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 561/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:54:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:54.408237 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca705bbc-a542-4413-8a24-ca578b9a1d88 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:54.511854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca705bbc-a542-4413-8a24-ca578b9a1d88 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:54.548779 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b0780-250b-496d-8f91-08c96e9091f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:54.549446 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b0780-250b-496d-8f91-08c96e9091f6 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:54.549730 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b0780-250b-496d-8f91-08c96e9091f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:54.550000 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b0780-250b-496d-8f91-08c96e9091f6 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:54.550713 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b0780-250b-496d-8f91-08c96e9091f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:54.552154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b0780-250b-496d-8f91-08c96e9091f6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:54.577395 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 549/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Jun 25 15:54:54 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:54:56.692272 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8df21-85b2-4d06-ae68-969d134418df None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:56.693340 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8df21-85b2-4d06-ae68-969d134418df None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:56.693340 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c8df21-85b2-4d06-ae68-969d134418df None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:56.695437 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 562/1111] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:56.703248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b5b2d-8ded-4ac3-83c6-bf723dc9d6a7 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:56.703699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b5b2d-8ded-4ac3-83c6-bf723dc9d6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:56.704167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b5b2d-8ded-4ac3-83c6-bf723dc9d6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:56.724087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c21b5b2d-8ded-4ac3-83c6-bf723dc9d6a7 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:56.725308 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c21b5b2d-8ded-4ac3-83c6-bf723dc9d6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:56.786700 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:56.823757 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 550/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:56.847470 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8df21-85b2-4d06-ae68-969d134418df None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:56.955172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c8df21-85b2-4d06-ae68-969d134418df None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:56.968967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:56.991671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be02d1b-d7b0-4880-a6c0-7c874bccdecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:56.992547 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be02d1b-d7b0-4880-a6c0-7c874bccdecb None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:56.992826 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be02d1b-d7b0-4880-a6c0-7c874bccdecb None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:56.993099 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be02d1b-d7b0-4880-a6c0-7c874bccdecb None admin] PATH_INFO: `/v3/roles/6322fb4a438d4a389c43d05248665e72` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:56.994593 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be02d1b-d7b0-4880-a6c0-7c874bccdecb None admin] RBAC: Authorizing `identity:get_role(role_id=6322fb4a438d4a389c43d05248665e72)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:56.995957 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be02d1b-d7b0-4880-a6c0-7c874bccdecb None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:56.997348 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 563/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:54:56 2022] GET /identity/v3/roles/6322fb4a438d4a389c43d05248665e72 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:57.003378 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b5b2d-8ded-4ac3-83c6-bf723dc9d6a7 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:57.109378 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b5b2d-8ded-4ac3-83c6-bf723dc9d6a7 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:57.148429 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c274402-159b-4835-9ecd-0be7ed7a8337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:57.149301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c274402-159b-4835-9ecd-0be7ed7a8337 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:57.149576 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c274402-159b-4835-9ecd-0be7ed7a8337 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:57.149845 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c274402-159b-4835-9ecd-0be7ed7a8337 None admin] PATH_INFO: `/v3/users/7804d07c1de94be7bbdd13deb6508564` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:57.163637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c274402-159b-4835-9ecd-0be7ed7a8337 None admin] RBAC: Authorizing `identity:get_user(user_id=7804d07c1de94be7bbdd13deb6508564)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:57.165112 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c274402-159b-4835-9ecd-0be7ed7a8337 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:57.166740 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 551/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:54:57 2022] GET /identity/v3/users/7804d07c1de94be7bbdd13deb6508564 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:57.173480 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be02d1b-d7b0-4880-a6c0-7c874bccdecb None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:57.283148 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be02d1b-d7b0-4880-a6c0-7c874bccdecb None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:57.318254 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e80dc-4b6f-4999-8e14-509bb685a0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:57.318885 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e80dc-4b6f-4999-8e14-509bb685a0dc None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:57.319165 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e80dc-4b6f-4999-8e14-509bb685a0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:57.319470 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e80dc-4b6f-4999-8e14-509bb685a0dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:57.370161 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:57.393932 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e80dc-4b6f-4999-8e14-509bb685a0dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:57.395390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e80dc-4b6f-4999-8e14-509bb685a0dc None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:57.502079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 564/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:54:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:54:57.510359 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c274402-159b-4835-9ecd-0be7ed7a8337 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:57.555354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:57.618676 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c274402-159b-4835-9ecd-0be7ed7a8337 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:57.653597 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f02ec-8986-4d90-961c-b63d98c83451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:57.655383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f02ec-8986-4d90-961c-b63d98c83451 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:57.655383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f02ec-8986-4d90-961c-b63d98c83451 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:57.655383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f02ec-8986-4d90-961c-b63d98c83451 None admin] PATH_INFO: `/v3/projects/6eabec0485e949329f72ca0f37f3d38b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:57.656136 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f02ec-8986-4d90-961c-b63d98c83451 None admin] RBAC: Authorizing `identity:get_project(project_id=6eabec0485e949329f72ca0f37f3d38b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:57.657592 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f02ec-8986-4d90-961c-b63d98c83451 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:57.659286 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 552/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:54:57 2022] GET /identity/v3/projects/6eabec0485e949329f72ca0f37f3d38b => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:57.666211 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e80dc-4b6f-4999-8e14-509bb685a0dc None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:54:57.767044 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:54:57.788525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e80dc-4b6f-4999-8e14-509bb685a0dc None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:54:57.824109 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ecf082-059d-4269-babe-7c63ee8a2518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:54:57.824706 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ecf082-059d-4269-babe-7c63ee8a2518 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:57.824979 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ecf082-059d-4269-babe-7c63ee8a2518 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:57.825249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ecf082-059d-4269-babe-7c63ee8a2518 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:57.826072 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ecf082-059d-4269-babe-7c63ee8a2518 None admin] RBAC: Adding query filter params (role.id=6322fb4a438d4a389c43d05248665e72, scope.project.id=6eabec0485e949329f72ca0f37f3d38b, user.id=7804d07c1de94be7bbdd13deb6508564) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:54:57.826384 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ecf082-059d-4269-babe-7c63ee8a2518 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:54:57.827819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ecf082-059d-4269-babe-7c63ee8a2518 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:54:57.833401 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 565/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:54:57 2022] GET /identity/v3/role_assignments?user.id=7804d07c1de94be7bbdd13deb6508564&scope.project.id=6eabec0485e949329f72ca0f37f3d38b&role.id=6322fb4a438d4a389c43d05248665e72 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:54:59.796576 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002c045-3dc2-4c3c-9929-169bcaf75d02 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:59.797370 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002c045-3dc2-4c3c-9929-169bcaf75d02 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:59.797370 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002c045-3dc2-4c3c-9929-169bcaf75d02 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:59.799710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 553/1118] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:54:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:54:59.805974 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc5087-ed91-434e-9ce9-f6a6e7d22348 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:54:59.806660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc5087-ed91-434e-9ce9-f6a6e7d22348 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:54:59.806760 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc5087-ed91-434e-9ce9-f6a6e7d22348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:54:59.828892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-00fc5087-ed91-434e-9ce9-f6a6e7d22348 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:54:59.830071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-00fc5087-ed91-434e-9ce9-f6a6e7d22348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:54:59.901666 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 566/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:54:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:54:59.924538 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002c045-3dc2-4c3c-9929-169bcaf75d02 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:00.040110 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002c045-3dc2-4c3c-9929-169bcaf75d02 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:00.047372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:00.077111 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07464d2d-33ba-48a0-826b-a081069619a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:00.077990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07464d2d-33ba-48a0-826b-a081069619a6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:00.078265 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07464d2d-33ba-48a0-826b-a081069619a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:00.078534 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07464d2d-33ba-48a0-826b-a081069619a6 None admin] PATH_INFO: `/v3/roles/6322fb4a438d4a389c43d05248665e72` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:00.080123 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07464d2d-33ba-48a0-826b-a081069619a6 None admin] RBAC: Authorizing `identity:get_role(role_id=6322fb4a438d4a389c43d05248665e72)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:00.081529 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07464d2d-33ba-48a0-826b-a081069619a6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:00.082881 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 554/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:54:59 2022] GET /identity/v3/roles/6322fb4a438d4a389c43d05248665e72 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:00.089505 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc5087-ed91-434e-9ce9-f6a6e7d22348 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:00.179894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:00.203653 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc5087-ed91-434e-9ce9-f6a6e7d22348 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:00.239943 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e014de-be13-4f6f-86b9-ee0e1c7da657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:00.240832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e014de-be13-4f6f-86b9-ee0e1c7da657 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:00.241109 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e014de-be13-4f6f-86b9-ee0e1c7da657 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:00.241434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e014de-be13-4f6f-86b9-ee0e1c7da657 None admin] PATH_INFO: `/v3/users/7804d07c1de94be7bbdd13deb6508564` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:00.242674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e014de-be13-4f6f-86b9-ee0e1c7da657 None admin] RBAC: Authorizing `identity:get_user(user_id=7804d07c1de94be7bbdd13deb6508564)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:00.244106 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e014de-be13-4f6f-86b9-ee0e1c7da657 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:00.245723 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 567/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:55:00 2022] GET /identity/v3/users/7804d07c1de94be7bbdd13deb6508564 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:00.251825 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07464d2d-33ba-48a0-826b-a081069619a6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:00.358897 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07464d2d-33ba-48a0-826b-a081069619a6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:00.395640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ed99d-8ee8-47d4-af03-acd59d16ab79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:00.396552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ed99d-8ee8-47d4-af03-acd59d16ab79 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:00.396823 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ed99d-8ee8-47d4-af03-acd59d16ab79 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:00.397092 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ed99d-8ee8-47d4-af03-acd59d16ab79 None admin] PATH_INFO: `/v3/projects/6eabec0485e949329f72ca0f37f3d38b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:00.398404 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9ed99d-8ee8-47d4-af03-acd59d16ab79 None admin] RBAC: Authorizing `identity:get_project(project_id=6eabec0485e949329f72ca0f37f3d38b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:00.399841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9ed99d-8ee8-47d4-af03-acd59d16ab79 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:00.401548 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 555/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:55:00 2022] GET /identity/v3/projects/6eabec0485e949329f72ca0f37f3d38b => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:00.410968 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e014de-be13-4f6f-86b9-ee0e1c7da657 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:00.521066 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e014de-be13-4f6f-86b9-ee0e1c7da657 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:00.556953 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d1f88-7f5a-445a-bb6a-a3caa30ace9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:00.557831 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d1f88-7f5a-445a-bb6a-a3caa30ace9c None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:00.558112 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d1f88-7f5a-445a-bb6a-a3caa30ace9c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:00.558382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d1f88-7f5a-445a-bb6a-a3caa30ace9c None admin] PATH_INFO: `/v3/projects/6eabec0485e949329f72ca0f37f3d38b/users/7804d07c1de94be7bbdd13deb6508564/roles/6322fb4a438d4a389c43d05248665e72` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:00.560985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d1f88-7f5a-445a-bb6a-a3caa30ace9c None admin] RBAC: Authorizing `identity:create_grant(project_id=6eabec0485e949329f72ca0f37f3d38b, user_id=7804d07c1de94be7bbdd13deb6508564, role_id=6322fb4a438d4a389c43d05248665e72)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:00.562407 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d1f88-7f5a-445a-bb6a-a3caa30ace9c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:00.578125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 568/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:55:00 2022] PUT /identity/v3/projects/6eabec0485e949329f72ca0f37f3d38b/users/7804d07c1de94be7bbdd13deb6508564/roles/6322fb4a438d4a389c43d05248665e72 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:55:02.571855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6830979-6f14-4888-a76b-6d872774b5e1 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:02.572476 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6830979-6f14-4888-a76b-6d872774b5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:02.572476 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6830979-6f14-4888-a76b-6d872774b5e1 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:02.574445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 556/1124] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:55:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:55:02.583358 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffbc166-d7d0-4f98-a291-8463b86e15a2 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:02.584066 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffbc166-d7d0-4f98-a291-8463b86e15a2 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:02.584209 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffbc166-d7d0-4f98-a291-8463b86e15a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:02.595132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:02.611510 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1ffbc166-d7d0-4f98-a291-8463b86e15a2 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:55:02.612824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1ffbc166-d7d0-4f98-a291-8463b86e15a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:55:02.682819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 569/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:55:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:55:02.707829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6830979-6f14-4888-a76b-6d872774b5e1 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:02.761687 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:02.823237 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6830979-6f14-4888-a76b-6d872774b5e1 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:02.882265 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58385dd0-a304-461d-a68d-3e71e2ff26de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:02.883516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58385dd0-a304-461d-a68d-3e71e2ff26de None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:02.883801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58385dd0-a304-461d-a68d-3e71e2ff26de None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:02.884070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58385dd0-a304-461d-a68d-3e71e2ff26de None admin] PATH_INFO: `/v3/roles/6322fb4a438d4a389c43d05248665e72` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:02.885855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58385dd0-a304-461d-a68d-3e71e2ff26de None admin] RBAC: Authorizing `identity:get_role(role_id=6322fb4a438d4a389c43d05248665e72)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:02.887260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58385dd0-a304-461d-a68d-3e71e2ff26de None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:02.888726 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 557/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:55:02 2022] GET /identity/v3/roles/6322fb4a438d4a389c43d05248665e72 => generated 223 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:02.898487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffbc166-d7d0-4f98-a291-8463b86e15a2 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:03.003057 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffbc166-d7d0-4f98-a291-8463b86e15a2 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:03.040452 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28706e-f439-4bd4-96fc-ba92f64868ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:03.041331 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28706e-f439-4bd4-96fc-ba92f64868ff None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:03.041604 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28706e-f439-4bd4-96fc-ba92f64868ff None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:03.041876 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28706e-f439-4bd4-96fc-ba92f64868ff None admin] PATH_INFO: `/v3/users/7804d07c1de94be7bbdd13deb6508564` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:03.043170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28706e-f439-4bd4-96fc-ba92f64868ff None admin] RBAC: Authorizing `identity:get_user(user_id=7804d07c1de94be7bbdd13deb6508564)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:03.044629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28706e-f439-4bd4-96fc-ba92f64868ff None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:03.046245 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 570/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:55:02 2022] GET /identity/v3/users/7804d07c1de94be7bbdd13deb6508564 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:03.054856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58385dd0-a304-461d-a68d-3e71e2ff26de None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:03.166366 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58385dd0-a304-461d-a68d-3e71e2ff26de None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:03.177003 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:03.202443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e67b75-6df6-4019-9e99-abbc089e184d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:03.203083 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e67b75-6df6-4019-9e99-abbc089e184d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:03.203360 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e67b75-6df6-4019-9e99-abbc089e184d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:03.203642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e67b75-6df6-4019-9e99-abbc089e184d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:03.275438 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e67b75-6df6-4019-9e99-abbc089e184d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:03.276911 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e67b75-6df6-4019-9e99-abbc089e184d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:03.376334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 558/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:55:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:55:03.385301 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28706e-f439-4bd4-96fc-ba92f64868ff None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:03.388568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:03.493636 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28706e-f439-4bd4-96fc-ba92f64868ff None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:03.528863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f5bcf-5b6d-416a-bbe5-a464b189c120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:03.529711 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f5bcf-5b6d-416a-bbe5-a464b189c120 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:03.529988 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f5bcf-5b6d-416a-bbe5-a464b189c120 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:03.530259 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f5bcf-5b6d-416a-bbe5-a464b189c120 None admin] PATH_INFO: `/v3/projects/6eabec0485e949329f72ca0f37f3d38b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:03.531554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f5bcf-5b6d-416a-bbe5-a464b189c120 None admin] RBAC: Authorizing `identity:get_project(project_id=6eabec0485e949329f72ca0f37f3d38b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:03.533002 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f5bcf-5b6d-416a-bbe5-a464b189c120 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:03.534696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 571/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:55:03 2022] GET /identity/v3/projects/6eabec0485e949329f72ca0f37f3d38b => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:03.542887 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e67b75-6df6-4019-9e99-abbc089e184d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:03.554487 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:03.645369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e67b75-6df6-4019-9e99-abbc089e184d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:03.683110 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21827a-1935-49d6-ab00-2d3207f21e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:03.683743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21827a-1935-49d6-ab00-2d3207f21e16 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:03.684013 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21827a-1935-49d6-ab00-2d3207f21e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:03.684289 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21827a-1935-49d6-ab00-2d3207f21e16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:03.685166 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21827a-1935-49d6-ab00-2d3207f21e16 None admin] RBAC: Adding query filter params (role.id=6322fb4a438d4a389c43d05248665e72, scope.project.id=6eabec0485e949329f72ca0f37f3d38b, user.id=7804d07c1de94be7bbdd13deb6508564) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:55:03.685465 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21827a-1935-49d6-ab00-2d3207f21e16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:03.686903 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21827a-1935-49d6-ab00-2d3207f21e16 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:03.692662 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 559/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:55:03 2022] GET /identity/v3/role_assignments?user.id=7804d07c1de94be7bbdd13deb6508564&scope.project.id=6eabec0485e949329f72ca0f37f3d38b&role.id=6322fb4a438d4a389c43d05248665e72 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:55:05.688682 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aa30f-c86b-4c19-867b-52d60922bf45 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:05.689576 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aa30f-c86b-4c19-867b-52d60922bf45 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:05.689576 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aa30f-c86b-4c19-867b-52d60922bf45 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:05.691784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 572/1131] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:55:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:55:05.697339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190da9f8-6200-4423-bf04-50ac93e8411d None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:05.697775 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190da9f8-6200-4423-bf04-50ac93e8411d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:05.697866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190da9f8-6200-4423-bf04-50ac93e8411d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:05.716029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-190da9f8-6200-4423-bf04-50ac93e8411d None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:55:05.717232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-190da9f8-6200-4423-bf04-50ac93e8411d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:55:05.785750 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 560/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:55:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:55:05.809506 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5aa30f-c86b-4c19-867b-52d60922bf45 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:05.892675 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:05.934921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5aa30f-c86b-4c19-867b-52d60922bf45 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:05.972924 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b80e4-650d-4090-9b6c-815358611e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:05.973643 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b80e4-650d-4090-9b6c-815358611e01 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:05.973922 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b80e4-650d-4090-9b6c-815358611e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:05.974191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b80e4-650d-4090-9b6c-815358611e01 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:05.975425 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b80e4-650d-4090-9b6c-815358611e01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:05.976896 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b80e4-650d-4090-9b6c-815358611e01 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:05.978529 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 573/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:55:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:05.984776 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190da9f8-6200-4423-bf04-50ac93e8411d None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:05.987930 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:06.090196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190da9f8-6200-4423-bf04-50ac93e8411d None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:06.126254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e91984-89b4-4208-938d-84801e94bf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:06.126894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e91984-89b4-4208-938d-84801e94bf16 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:06.127167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e91984-89b4-4208-938d-84801e94bf16 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:06.127436 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e91984-89b4-4208-938d-84801e94bf16 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:06.128130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e91984-89b4-4208-938d-84801e94bf16 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:06.129588 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e91984-89b4-4208-938d-84801e94bf16 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:06.158669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 561/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Jun 25 15:55:05 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:55:08.184330 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc84515-4f35-4a52-af6b-733bbc268788 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:08.185196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc84515-4f35-4a52-af6b-733bbc268788 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:08.185306 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc84515-4f35-4a52-af6b-733bbc268788 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:08.187475 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 574/1135] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:55:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:55:08.192928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a152c04-98b7-4328-b071-06e3e5feb734 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:08.193355 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a152c04-98b7-4328-b071-06e3e5feb734 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:08.193446 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a152c04-98b7-4328-b071-06e3e5feb734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:08.212260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2a152c04-98b7-4328-b071-06e3e5feb734 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:55:08.213546 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2a152c04-98b7-4328-b071-06e3e5feb734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:55:08.283338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:08.302492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 562/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:55:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:55:08.327493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc84515-4f35-4a52-af6b-733bbc268788 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:08.452704 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc84515-4f35-4a52-af6b-733bbc268788 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:08.487400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255918cf-9bb2-4357-9066-00016f5dab43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:08.488184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255918cf-9bb2-4357-9066-00016f5dab43 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:08.488448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255918cf-9bb2-4357-9066-00016f5dab43 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:08.488746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255918cf-9bb2-4357-9066-00016f5dab43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:08.490028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255918cf-9bb2-4357-9066-00016f5dab43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:08.491445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255918cf-9bb2-4357-9066-00016f5dab43 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:08.493222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 575/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:55:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:08.499089 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a152c04-98b7-4328-b071-06e3e5feb734 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:08.605583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a152c04-98b7-4328-b071-06e3e5feb734 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:08.643066 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8769777-0224-4e5a-b384-5bd637248c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:08.643721 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8769777-0224-4e5a-b384-5bd637248c11 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:08.643998 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8769777-0224-4e5a-b384-5bd637248c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:08.644270 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8769777-0224-4e5a-b384-5bd637248c11 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:08.645012 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8769777-0224-4e5a-b384-5bd637248c11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:08.646448 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8769777-0224-4e5a-b384-5bd637248c11 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:08.668262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 563/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Jun 25 15:55:08 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:55:10.708817 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484fdc8-1ce6-47f6-979e-1840466b0b11 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:10.709658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484fdc8-1ce6-47f6-979e-1840466b0b11 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:10.709658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484fdc8-1ce6-47f6-979e-1840466b0b11 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:10.711881 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 576/1139] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:55:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:55:10.717460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becea1f6-70ba-44b8-9877-4237a544f77d None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:10.717871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becea1f6-70ba-44b8-9877-4237a544f77d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:10.718041 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becea1f6-70ba-44b8-9877-4237a544f77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:10.735344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-becea1f6-70ba-44b8-9877-4237a544f77d None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:55:10.736596 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-becea1f6-70ba-44b8-9877-4237a544f77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:55:10.782309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:10.815788 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 564/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:55:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:55:10.839728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484fdc8-1ce6-47f6-979e-1840466b0b11 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:10.849674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:10.944607 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484fdc8-1ce6-47f6-979e-1840466b0b11 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:10.983418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb11a83-bbea-4f8c-9640-2c87ab7c877f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:10.984133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb11a83-bbea-4f8c-9640-2c87ab7c877f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:10.984453 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb11a83-bbea-4f8c-9640-2c87ab7c877f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:10.984721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb11a83-bbea-4f8c-9640-2c87ab7c877f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:10.995400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb11a83-bbea-4f8c-9640-2c87ab7c877f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:55:11.002906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-8eb11a83-bbea-4f8c-9640-2c87ab7c877f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:55:11.002906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:55:11.002906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:55:11.002906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:55:11.002906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:55:11.002906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:55:11.002906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:55:11.002906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:55:11.002906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:55:11.002906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:55:11.002906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:55:11.002906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:55:11.002906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:55:11.002906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:55:11.002906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:55:11.002906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:55:11.004053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:55:11.004053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:55:11.004053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:55:11.004053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:55:11.004053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:55:11.004053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:55:11.004053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:55:11.004053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:55:11.004053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:55:11.004053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:55:11.004053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:55:11.004053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:55:11.004053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:55:11.004053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:55:11.004053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:55:11.004053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:55:11.005243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:55:11.005243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:55:11.005243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:55:11.005243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:55:11.005243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:55:11.005243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:55:11.005243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:55:11.005243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:55:11.005243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:55:11.005243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:55:11.005243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:55:11.005243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:55:11.005243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 577/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:55:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:55:11.010901 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becea1f6-70ba-44b8-9877-4237a544f77d None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:11.122296 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becea1f6-70ba-44b8-9877-4237a544f77d None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:11.168395 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8095d5a-97e4-484a-b34d-0b636384e691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:11.169079 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8095d5a-97e4-484a-b34d-0b636384e691 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:11.169357 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8095d5a-97e4-484a-b34d-0b636384e691 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:11.169625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8095d5a-97e4-484a-b34d-0b636384e691 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:11.170363 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8095d5a-97e4-484a-b34d-0b636384e691 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:55:11.170666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8095d5a-97e4-484a-b34d-0b636384e691 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:11.172045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8095d5a-97e4-484a-b34d-0b636384e691 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:11.180782 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 565/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:55:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:11.189774 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb11a83-bbea-4f8c-9640-2c87ab7c877f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:11.267458 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:11.310941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb11a83-bbea-4f8c-9640-2c87ab7c877f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:11.345640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13de235-b736-415f-b8b7-12df7040f8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:11.346499 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13de235-b736-415f-b8b7-12df7040f8dd None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:11.346773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13de235-b736-415f-b8b7-12df7040f8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:11.347047 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13de235-b736-415f-b8b7-12df7040f8dd None admin] PATH_INFO: `/v3/users/b7b1da909bcd4cc7a970ef211df2c9b6` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:11.360626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13de235-b736-415f-b8b7-12df7040f8dd None admin] RBAC: Authorizing `identity:get_user(user_id=b7b1da909bcd4cc7a970ef211df2c9b6)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:11.362069 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13de235-b736-415f-b8b7-12df7040f8dd None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:11.363707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 578/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:55:11 2022] GET /identity/v3/users/b7b1da909bcd4cc7a970ef211df2c9b6 => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:11.371080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8095d5a-97e4-484a-b34d-0b636384e691 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:11.406801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:11.475492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8095d5a-97e4-484a-b34d-0b636384e691 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:11.514312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3599a44-be01-465d-ae05-e6b31ccb11ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:11.514937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3599a44-be01-465d-ae05-e6b31ccb11ee None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:11.515211 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3599a44-be01-465d-ae05-e6b31ccb11ee None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:11.515482 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3599a44-be01-465d-ae05-e6b31ccb11ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:11.587117 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3599a44-be01-465d-ae05-e6b31ccb11ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:11.588502 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3599a44-be01-465d-ae05-e6b31ccb11ee None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:11.625056 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:11.697848 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 566/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:55:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:55:11.704776 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13de235-b736-415f-b8b7-12df7040f8dd None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:11.825828 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13de235-b736-415f-b8b7-12df7040f8dd None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:11.859851 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:11.889830 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb09032-bdb7-40f5-b9d3-6d076a46af9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:11.890688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb09032-bdb7-40f5-b9d3-6d076a46af9c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:11.890967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb09032-bdb7-40f5-b9d3-6d076a46af9c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:11.891558 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb09032-bdb7-40f5-b9d3-6d076a46af9c None admin] PATH_INFO: `/v3/projects/5d4deb47c1d34818a1a420f3d675fa2b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:11.892821 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb09032-bdb7-40f5-b9d3-6d076a46af9c None admin] RBAC: Authorizing `identity:get_project(project_id=5d4deb47c1d34818a1a420f3d675fa2b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:11.894255 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb09032-bdb7-40f5-b9d3-6d076a46af9c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:11.895963 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 579/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:55:11 2022] GET /identity/v3/projects/5d4deb47c1d34818a1a420f3d675fa2b => generated 311 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:11.902720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3599a44-be01-465d-ae05-e6b31ccb11ee None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:12.005591 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3599a44-be01-465d-ae05-e6b31ccb11ee None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:12.040873 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bab578-c327-4d4a-82aa-cf8462dcf1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:12.041511 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bab578-c327-4d4a-82aa-cf8462dcf1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:12.041788 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bab578-c327-4d4a-82aa-cf8462dcf1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:12.042054 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bab578-c327-4d4a-82aa-cf8462dcf1d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:12.042876 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bab578-c327-4d4a-82aa-cf8462dcf1d0 None admin] RBAC: Adding query filter params (role.id=b1e13c449fa64b3e88b6832bb767214b, scope.project.id=5d4deb47c1d34818a1a420f3d675fa2b, user.id=b7b1da909bcd4cc7a970ef211df2c9b6) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:55:12.043178 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bab578-c327-4d4a-82aa-cf8462dcf1d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:12.044627 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bab578-c327-4d4a-82aa-cf8462dcf1d0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:12.046497 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:12.051275 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 567/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:55:11 2022] GET /identity/v3/role_assignments?user.id=b7b1da909bcd4cc7a970ef211df2c9b6&scope.project.id=5d4deb47c1d34818a1a420f3d675fa2b&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:14.022140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aea138-fcb9-49a3-88e5-aad0c07bc5d2 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:14.022997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aea138-fcb9-49a3-88e5-aad0c07bc5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:14.022997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aea138-fcb9-49a3-88e5-aad0c07bc5d2 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:14.025354 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 580/1147] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:55:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:55:14.034306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64abf8-70d6-4103-96c8-4e7869bb7f67 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:14.034761 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64abf8-70d6-4103-96c8-4e7869bb7f67 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:14.034855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64abf8-70d6-4103-96c8-4e7869bb7f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:14.054574 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5c64abf8-70d6-4103-96c8-4e7869bb7f67 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:55:14.055883 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5c64abf8-70d6-4103-96c8-4e7869bb7f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:55:14.135580 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 568/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:55:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:55:14.159401 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aea138-fcb9-49a3-88e5-aad0c07bc5d2 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:14.277835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aea138-fcb9-49a3-88e5-aad0c07bc5d2 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:14.317093 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce21713-9d9b-4d31-be71-3ab51485e7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:14.317886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce21713-9d9b-4d31-be71-3ab51485e7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:14.318167 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce21713-9d9b-4d31-be71-3ab51485e7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:14.318445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce21713-9d9b-4d31-be71-3ab51485e7c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:14.330865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce21713-9d9b-4d31-be71-3ab51485e7c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:55:14.339074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-bce21713-9d9b-4d31-be71-3ab51485e7c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:55:14.339074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:55:14.339074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:55:14.339074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:55:14.339074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:55:14.339074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:55:14.339074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:55:14.339074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:55:14.339074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:55:14.339074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:55:14.339074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:55:14.339074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:55:14.339074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:55:14.339074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:55:14.339074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:55:14.339074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:55:14.340162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:55:14.340162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:55:14.340162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:55:14.340162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:55:14.340162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:55:14.340162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:55:14.340162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:55:14.340162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:55:14.340162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:55:14.340162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:55:14.340162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:55:14.340162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:55:14.340162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:55:14.340162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:55:14.340162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:55:14.340162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:55:14.341317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:55:14.341317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:55:14.341317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:55:14.341317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:55:14.341317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:55:14.341317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:55:14.341317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:55:14.341317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:55:14.341317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:55:14.341317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:55:14.341317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:55:14.341317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:55:14.341317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 581/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:55:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:55:14.346874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64abf8-70d6-4103-96c8-4e7869bb7f67 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:14.465598 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64abf8-70d6-4103-96c8-4e7869bb7f67 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:14.492711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:14.503105 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a96ab0a-d88c-44f1-ac1f-39f19b8780b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:14.503766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96ab0a-d88c-44f1-ac1f-39f19b8780b9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:14.504048 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96ab0a-d88c-44f1-ac1f-39f19b8780b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:14.504322 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96ab0a-d88c-44f1-ac1f-39f19b8780b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:14.505106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a96ab0a-d88c-44f1-ac1f-39f19b8780b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:55:14.505415 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a96ab0a-d88c-44f1-ac1f-39f19b8780b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:14.506800 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a96ab0a-d88c-44f1-ac1f-39f19b8780b9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:14.515130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 569/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:55:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:14.524383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce21713-9d9b-4d31-be71-3ab51485e7c7 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:14.580721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:14.641432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce21713-9d9b-4d31-be71-3ab51485e7c7 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:14.676640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb962a0d-f924-40c2-a4d0-637af655fe37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:14.677510 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb962a0d-f924-40c2-a4d0-637af655fe37 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:14.677791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb962a0d-f924-40c2-a4d0-637af655fe37 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:14.678071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb962a0d-f924-40c2-a4d0-637af655fe37 None admin] PATH_INFO: `/v3/users/b7b1da909bcd4cc7a970ef211df2c9b6` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:14.679310 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb962a0d-f924-40c2-a4d0-637af655fe37 None admin] RBAC: Authorizing `identity:get_user(user_id=b7b1da909bcd4cc7a970ef211df2c9b6)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:14.680931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb962a0d-f924-40c2-a4d0-637af655fe37 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:14.682564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 582/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:55:14 2022] GET /identity/v3/users/b7b1da909bcd4cc7a970ef211df2c9b6 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:14.691370 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a96ab0a-d88c-44f1-ac1f-39f19b8780b9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:14.804981 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a96ab0a-d88c-44f1-ac1f-39f19b8780b9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:14.841299 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7e95e-3c88-4f7d-82d0-f82e831f1a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:14.842159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7e95e-3c88-4f7d-82d0-f82e831f1a84 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:14.842432 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7e95e-3c88-4f7d-82d0-f82e831f1a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:14.842702 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7e95e-3c88-4f7d-82d0-f82e831f1a84 None admin] PATH_INFO: `/v3/projects/5d4deb47c1d34818a1a420f3d675fa2b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:14.844020 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7e95e-3c88-4f7d-82d0-f82e831f1a84 None admin] RBAC: Authorizing `identity:get_project(project_id=5d4deb47c1d34818a1a420f3d675fa2b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:14.845502 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7e95e-3c88-4f7d-82d0-f82e831f1a84 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:14.847191 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 570/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:55:14 2022] GET /identity/v3/projects/5d4deb47c1d34818a1a420f3d675fa2b => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:55:14.853367 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb962a0d-f924-40c2-a4d0-637af655fe37 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:14.958467 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb962a0d-f924-40c2-a4d0-637af655fe37 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:14.977996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:14.995370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fbbfc-aaf9-4dbe-ab39-07fde7f333b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:14.996294 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fbbfc-aaf9-4dbe-ab39-07fde7f333b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:14.996603 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fbbfc-aaf9-4dbe-ab39-07fde7f333b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:14.996874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fbbfc-aaf9-4dbe-ab39-07fde7f333b1 None admin] PATH_INFO: `/v3/projects/5d4deb47c1d34818a1a420f3d675fa2b/users/b7b1da909bcd4cc7a970ef211df2c9b6/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:14.998968 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22fbbfc-aaf9-4dbe-ab39-07fde7f333b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d4deb47c1d34818a1a420f3d675fa2b, user_id=b7b1da909bcd4cc7a970ef211df2c9b6, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:15.000542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22fbbfc-aaf9-4dbe-ab39-07fde7f333b1 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:15.015274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 583/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:55:14 2022] PUT /identity/v3/projects/5d4deb47c1d34818a1a420f3d675fa2b/users/b7b1da909bcd4cc7a970ef211df2c9b6/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:55:17.002048 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f99e8-0c6c-4328-a851-b8dccf91ad8d None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:17.002929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f99e8-0c6c-4328-a851-b8dccf91ad8d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:17.002929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f99e8-0c6c-4328-a851-b8dccf91ad8d None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:17.005191 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 571/1154] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:55:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:55:17.011760 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff042c9-62f2-4f8e-abf5-c1d8422ad1c1 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:17.012211 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff042c9-62f2-4f8e-abf5-c1d8422ad1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:17.012329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff042c9-62f2-4f8e-abf5-c1d8422ad1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:17.039988 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-dff042c9-62f2-4f8e-abf5-c1d8422ad1c1 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:55:17.041203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-dff042c9-62f2-4f8e-abf5-c1d8422ad1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:55:17.110773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 584/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:55:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:55:17.139853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f99e8-0c6c-4328-a851-b8dccf91ad8d None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:17.206285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:17.254500 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f99e8-0c6c-4328-a851-b8dccf91ad8d None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:17.289512 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2682e19-cfc8-4718-8ca6-1eb5d3e4239e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:17.290354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2682e19-cfc8-4718-8ca6-1eb5d3e4239e None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:17.290631 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2682e19-cfc8-4718-8ca6-1eb5d3e4239e None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:17.290894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2682e19-cfc8-4718-8ca6-1eb5d3e4239e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:17.303194 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2682e19-cfc8-4718-8ca6-1eb5d3e4239e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:55:17.310894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-b2682e19-cfc8-4718-8ca6-1eb5d3e4239e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:55:17.310894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:55:17.310894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:55:17.310894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:55:17.310894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:55:17.310894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:55:17.310894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:55:17.310894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:55:17.310894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:55:17.310894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:55:17.310894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:55:17.310894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:55:17.310894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:55:17.310894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:55:17.310894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:55:17.310894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:55:17.311962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:55:17.311962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:55:17.311962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:55:17.311962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:55:17.311962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:55:17.311962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:55:17.311962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:55:17.311962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:55:17.311962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:55:17.311962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:55:17.311962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:55:17.311962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:55:17.311962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:55:17.311962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:55:17.311962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:55:17.311962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:55:17.313415 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:55:17.313415 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:55:17.313415 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:55:17.313415 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:55:17.313415 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:55:17.313415 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:55:17.313415 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:55:17.313415 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:55:17.313415 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:55:17.313415 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:55:17.313415 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:55:17.313415 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:55:17.313415 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 572/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:55:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:55:17.321877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff042c9-62f2-4f8e-abf5-c1d8422ad1c1 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:17.426340 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff042c9-62f2-4f8e-abf5-c1d8422ad1c1 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:17.436807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:17.464726 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8addbc5e-8813-4c24-b55e-d64db62624f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:17.465377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addbc5e-8813-4c24-b55e-d64db62624f8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:17.465650 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addbc5e-8813-4c24-b55e-d64db62624f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:17.465921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addbc5e-8813-4c24-b55e-d64db62624f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:17.466659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8addbc5e-8813-4c24-b55e-d64db62624f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:55:17.466958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8addbc5e-8813-4c24-b55e-d64db62624f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:17.468317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8addbc5e-8813-4c24-b55e-d64db62624f8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:17.476384 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 585/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:55:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:17.482496 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2682e19-cfc8-4718-8ca6-1eb5d3e4239e None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:17.595910 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2682e19-cfc8-4718-8ca6-1eb5d3e4239e None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:17.616259 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:17.634725 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69091eff-23f6-48c2-9df8-259e325d8535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:17.635588 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69091eff-23f6-48c2-9df8-259e325d8535 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:17.635868 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69091eff-23f6-48c2-9df8-259e325d8535 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:17.636129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69091eff-23f6-48c2-9df8-259e325d8535 None admin] PATH_INFO: `/v3/users/b7b1da909bcd4cc7a970ef211df2c9b6` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:17.637411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69091eff-23f6-48c2-9df8-259e325d8535 None admin] RBAC: Authorizing `identity:get_user(user_id=b7b1da909bcd4cc7a970ef211df2c9b6)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:17.638851 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69091eff-23f6-48c2-9df8-259e325d8535 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:17.640482 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 573/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:55:17 2022] GET /identity/v3/users/b7b1da909bcd4cc7a970ef211df2c9b6 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:17.646830 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8addbc5e-8813-4c24-b55e-d64db62624f8 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:17.760618 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8addbc5e-8813-4c24-b55e-d64db62624f8 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:17.798217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fae8e6-6e6b-400e-805f-b3243cf0d24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:17.798856 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fae8e6-6e6b-400e-805f-b3243cf0d24b None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:17.799130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fae8e6-6e6b-400e-805f-b3243cf0d24b None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:17.799406 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fae8e6-6e6b-400e-805f-b3243cf0d24b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:17.840202 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:17.869844 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fae8e6-6e6b-400e-805f-b3243cf0d24b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:17.871318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fae8e6-6e6b-400e-805f-b3243cf0d24b None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:17.975605 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 586/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:55:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:55:17.984434 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69091eff-23f6-48c2-9df8-259e325d8535 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:18.092847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69091eff-23f6-48c2-9df8-259e325d8535 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:18.134946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1010b79a-bcec-4516-90b5-c31bfe4e25e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:18.135811 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010b79a-bcec-4516-90b5-c31bfe4e25e6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:18.136095 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010b79a-bcec-4516-90b5-c31bfe4e25e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:18.136365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010b79a-bcec-4516-90b5-c31bfe4e25e6 None admin] PATH_INFO: `/v3/projects/5d4deb47c1d34818a1a420f3d675fa2b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:18.137731 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1010b79a-bcec-4516-90b5-c31bfe4e25e6 None admin] RBAC: Authorizing `identity:get_project(project_id=5d4deb47c1d34818a1a420f3d675fa2b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:18.139190 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1010b79a-bcec-4516-90b5-c31bfe4e25e6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:18.140950 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 574/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:55:17 2022] GET /identity/v3/projects/5d4deb47c1d34818a1a420f3d675fa2b => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:55:18.150447 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fae8e6-6e6b-400e-805f-b3243cf0d24b None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:18.227591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:18.257448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fae8e6-6e6b-400e-805f-b3243cf0d24b None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:18.293191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa7707-6b38-4fa3-ab67-503c44dcec3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:18.293831 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa7707-6b38-4fa3-ab67-503c44dcec3e None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:18.294112 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa7707-6b38-4fa3-ab67-503c44dcec3e None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:18.294380 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa7707-6b38-4fa3-ab67-503c44dcec3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:18.295212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa7707-6b38-4fa3-ab67-503c44dcec3e None admin] RBAC: Adding query filter params (role.id=b1e13c449fa64b3e88b6832bb767214b, scope.project.id=5d4deb47c1d34818a1a420f3d675fa2b, user.id=b7b1da909bcd4cc7a970ef211df2c9b6) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:55:18.295519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa7707-6b38-4fa3-ab67-503c44dcec3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:18.296973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa7707-6b38-4fa3-ab67-503c44dcec3e None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:18.303234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 587/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:55:18 2022] GET /identity/v3/role_assignments?user.id=b7b1da909bcd4cc7a970ef211df2c9b6&scope.project.id=5d4deb47c1d34818a1a420f3d675fa2b&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:20.317443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e22495-46d4-4e86-b453-f82440b9ad8b None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:20.318284 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e22495-46d4-4e86-b453-f82440b9ad8b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:20.318284 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e22495-46d4-4e86-b453-f82440b9ad8b None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:20.320532 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 575/1162] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:55:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:55:20.329548 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df72db-5292-4d02-851a-da05619bdfa6 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:20.330051 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df72db-5292-4d02-851a-da05619bdfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:20.330208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df72db-5292-4d02-851a-da05619bdfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:20.350816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a5df72db-5292-4d02-851a-da05619bdfa6 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:55:20.352307 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a5df72db-5292-4d02-851a-da05619bdfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:55:20.423050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 588/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:55:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:55:20.448265 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e22495-46d4-4e86-b453-f82440b9ad8b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:20.514185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:20.564782 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e22495-46d4-4e86-b453-f82440b9ad8b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:20.599907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831421c6-c56d-4148-8c4d-f9cd18904d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:20.600509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831421c6-c56d-4148-8c4d-f9cd18904d2d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:20.600793 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831421c6-c56d-4148-8c4d-f9cd18904d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:20.601065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831421c6-c56d-4148-8c4d-f9cd18904d2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:20.675856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831421c6-c56d-4148-8c4d-f9cd18904d2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:20.677403 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831421c6-c56d-4148-8c4d-f9cd18904d2d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:20.733315 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:20.786044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 576/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:55:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:55:20.792986 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df72db-5292-4d02-851a-da05619bdfa6 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:20.872859 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:20.909737 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df72db-5292-4d02-851a-da05619bdfa6 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:20.954654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d43b7-8beb-4df0-afad-0d0c54c453fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:20.955770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d43b7-8beb-4df0-afad-0d0c54c453fd None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:20.956264 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d43b7-8beb-4df0-afad-0d0c54c453fd None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:20.956917 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d43b7-8beb-4df0-afad-0d0c54c453fd None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:20.966990 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d43b7-8beb-4df0-afad-0d0c54c453fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:20.968446 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0d43b7-8beb-4df0-afad-0d0c54c453fd None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:20.983175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-bd0d43b7-8beb-4df0-afad-0d0c54c453fd None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 25 15:55:20.983175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:55:20.983175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 25 15:55:20.983175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 25 15:55:20.983175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:55:20.983175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:55:20.983175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:55:20.983175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:55:20.983175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 25 15:55:20.983175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:55:20.983175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:55:20.983175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:55:20.983175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:55:20.983175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:55:20.983175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:55:20.983175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:55:20.983175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:55:20.984344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:55:20.984344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:55:20.984344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:55:20.984344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:55:20.984344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:55:20.984344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:55:20.984344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:55:20.984344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:55:20.984344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:55:20.984344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:55:20.984344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:55:20.984344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:55:20.984344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:55:20.984344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:55:20.984344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:55:20.984344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:55:20.985242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:55:20.985242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:55:20.985242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:55:20.985242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:55:20.985242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:55:20.985242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:55:20.985242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:55:20.985242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:55:20.985242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:55:20.985242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:55:20.985242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 25 15:55:20.985242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:55:20.985242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 25 15:55:20.985242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:55:20.985242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 589/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Jun 25 15:55:20 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:55:20.991260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831421c6-c56d-4148-8c4d-f9cd18904d2d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:21.102556 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831421c6-c56d-4148-8c4d-f9cd18904d2d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:21.139567 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce3d02-02da-4626-b892-c72b6af6f9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:21.140144 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce3d02-02da-4626-b892-c72b6af6f9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:21.140440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce3d02-02da-4626-b892-c72b6af6f9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:21.140716 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce3d02-02da-4626-b892-c72b6af6f9b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:21.141756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce3d02-02da-4626-b892-c72b6af6f9b6 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:55:21.142053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce3d02-02da-4626-b892-c72b6af6f9b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:21.143431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce3d02-02da-4626-b892-c72b6af6f9b6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:21.151197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 577/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Sat Jun 25 15:55:20 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:23.110585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c94a5-4fa9-499c-bdd4-33aa8af44eb3 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:23.111129 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c94a5-4fa9-499c-bdd4-33aa8af44eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:23.111129 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c94a5-4fa9-499c-bdd4-33aa8af44eb3 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:23.112436 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 590/1167] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:55:23.119035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ac169-82dd-4cc8-aa7b-d7d7caa89905 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:23.119740 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ac169-82dd-4cc8-aa7b-d7d7caa89905 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:23.119892 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ac169-82dd-4cc8-aa7b-d7d7caa89905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:23.150158 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-568ac169-82dd-4cc8-aa7b-d7d7caa89905 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:55:23.151980 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-568ac169-82dd-4cc8-aa7b-d7d7caa89905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:55:23.170837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:23.228345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 578/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:55:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:55:23.251482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c94a5-4fa9-499c-bdd4-33aa8af44eb3 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:23.366846 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c94a5-4fa9-499c-bdd4-33aa8af44eb3 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:23.393834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:23.407479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef9b4f-35ee-4972-9a96-208b927b59b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:23.408611 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef9b4f-35ee-4972-9a96-208b927b59b1 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:23.409089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef9b4f-35ee-4972-9a96-208b927b59b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:23.409546 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef9b4f-35ee-4972-9a96-208b927b59b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:23.410789 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef9b4f-35ee-4972-9a96-208b927b59b1 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:23.413070 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef9b4f-35ee-4972-9a96-208b927b59b1 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:23.436190 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 591/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Sat Jun 25 15:55:23 2022] POST /identity/v3/domains => generated 267 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:55:25.484029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aef079-55f3-4f31-b6c2-efaf55251c15 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:25.485109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aef079-55f3-4f31-b6c2-efaf55251c15 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:25.485109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aef079-55f3-4f31-b6c2-efaf55251c15 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:25.487191 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 579/1170] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:55:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:55:25.494733 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8c6d0-27a3-4bc9-a995-4da7139f1c57 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:25.495155 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8c6d0-27a3-4bc9-a995-4da7139f1c57 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:25.495261 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8c6d0-27a3-4bc9-a995-4da7139f1c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:25.512650 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-48a8c6d0-27a3-4bc9-a995-4da7139f1c57 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:55:25.513838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-48a8c6d0-27a3-4bc9-a995-4da7139f1c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:55:25.586621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 592/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:55:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:55:25.618921 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aef079-55f3-4f31-b6c2-efaf55251c15 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:25.737938 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aef079-55f3-4f31-b6c2-efaf55251c15 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:25.775775 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f304fc-f19e-4793-b29b-5f6aba5f4952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:25.776670 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f304fc-f19e-4793-b29b-5f6aba5f4952 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:25.776952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f304fc-f19e-4793-b29b-5f6aba5f4952 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:25.777224 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f304fc-f19e-4793-b29b-5f6aba5f4952 None admin] PATH_INFO: `/v3/domains/c729806c815049d79106badea7abbb28` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:25.788869 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f304fc-f19e-4793-b29b-5f6aba5f4952 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c729806c815049d79106badea7abbb28)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:25.790357 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f304fc-f19e-4793-b29b-5f6aba5f4952 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:25.791988 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 580/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Sat Jun 25 15:55:25 2022] GET /identity/v3/domains/c729806c815049d79106badea7abbb28 => generated 267 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:25.798941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8c6d0-27a3-4bc9-a995-4da7139f1c57 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:25.905686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8c6d0-27a3-4bc9-a995-4da7139f1c57 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:25.923205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:25.943887 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37464232-349a-4f7c-8f1c-af8cb5d409f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:25.944565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37464232-349a-4f7c-8f1c-af8cb5d409f7 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:25.944845 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37464232-349a-4f7c-8f1c-af8cb5d409f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:25.945151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37464232-349a-4f7c-8f1c-af8cb5d409f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:25.945863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37464232-349a-4f7c-8f1c-af8cb5d409f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:25.947289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37464232-349a-4f7c-8f1c-af8cb5d409f7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:25.976694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 593/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Jun 25 15:55:25 2022] POST /identity/v3/projects => generated 361 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:55:28.103680 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c0dd0-07b8-48dd-8991-19a1ba45780e None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:28.104288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c0dd0-07b8-48dd-8991-19a1ba45780e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:28.104288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c0dd0-07b8-48dd-8991-19a1ba45780e None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:28.105656 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 581/1174] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:55:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:55:28.112218 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1af148-4227-4c70-ad9d-373b7a1d5aed None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:28.113008 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1af148-4227-4c70-ad9d-373b7a1d5aed None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:28.113139 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1af148-4227-4c70-ad9d-373b7a1d5aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:28.138513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-2b1af148-4227-4c70-ad9d-373b7a1d5aed None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:55:28.139718 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2b1af148-4227-4c70-ad9d-373b7a1d5aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:55:28.207443 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 594/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:55:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:55:28.239902 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c0dd0-07b8-48dd-8991-19a1ba45780e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:28.251836 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:28.354473 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c0dd0-07b8-48dd-8991-19a1ba45780e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:28.392079 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a3f65-b74d-44ca-8d8d-0d157b7fa7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:28.392904 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a3f65-b74d-44ca-8d8d-0d157b7fa7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:28.393184 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a3f65-b74d-44ca-8d8d-0d157b7fa7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:28.393457 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a3f65-b74d-44ca-8d8d-0d157b7fa7f9 None admin] PATH_INFO: `/v3/domains/c729806c815049d79106badea7abbb28` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:28.394676 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a3f65-b74d-44ca-8d8d-0d157b7fa7f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c729806c815049d79106badea7abbb28)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:28.396118 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a3f65-b74d-44ca-8d8d-0d157b7fa7f9 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:28.397786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 582/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Sat Jun 25 15:55:28 2022] GET /identity/v3/domains/c729806c815049d79106badea7abbb28 => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:28.404575 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1af148-4227-4c70-ad9d-373b7a1d5aed None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:28.508593 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:28.513992 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1af148-4227-4c70-ad9d-373b7a1d5aed None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:28.555112 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8d135-3c72-48c0-8553-2668010c24e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:28.555777 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8d135-3c72-48c0-8553-2668010c24e8 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:28.556054 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8d135-3c72-48c0-8553-2668010c24e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:28.556312 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8d135-3c72-48c0-8553-2668010c24e8 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:28.557041 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8d135-3c72-48c0-8553-2668010c24e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:28.558475 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8d135-3c72-48c0-8553-2668010c24e8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:28.585949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 595/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Jun 25 15:55:28 2022] POST /identity/v3/users => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:55:30.717845 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda76e1-4096-456e-9a36-f05d03bc23eb None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:30.718746 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda76e1-4096-456e-9a36-f05d03bc23eb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:30.718746 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda76e1-4096-456e-9a36-f05d03bc23eb None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:30.720987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 583/1178] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:55:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:55:30.726342 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c65086-cc3c-41d6-881b-ced6f7b5a21c None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:30.726768 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c65086-cc3c-41d6-881b-ced6f7b5a21c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:30.726869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c65086-cc3c-41d6-881b-ced6f7b5a21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:30.745403 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b9c65086-cc3c-41d6-881b-ced6f7b5a21c None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:55:30.746572 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b9c65086-cc3c-41d6-881b-ced6f7b5a21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:55:30.813696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 596/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:55:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:55:30.837570 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda76e1-4096-456e-9a36-f05d03bc23eb None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:30.931868 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:30.957148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda76e1-4096-456e-9a36-f05d03bc23eb None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:30.991759 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa56a2-1626-400c-89cb-40d06094f5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:30.992774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa56a2-1626-400c-89cb-40d06094f5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:30.993056 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa56a2-1626-400c-89cb-40d06094f5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:30.993326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa56a2-1626-400c-89cb-40d06094f5a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:31.005173 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa56a2-1626-400c-89cb-40d06094f5a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:55:31.012912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-d0aa56a2-1626-400c-89cb-40d06094f5a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:55:31.012912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:55:31.012912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:55:31.012912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:55:31.012912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:55:31.012912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:55:31.012912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:55:31.012912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:55:31.012912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:55:31.012912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:55:31.012912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:55:31.012912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:55:31.012912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:55:31.012912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:55:31.012912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:55:31.012912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:55:31.014070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:55:31.014070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:55:31.014070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:55:31.014070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:55:31.014070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:55:31.014070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:55:31.014070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:55:31.014070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:55:31.014070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:55:31.014070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:55:31.014070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:55:31.014070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:55:31.014070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:55:31.014070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:55:31.014070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:55:31.014070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:55:31.016875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:55:31.016875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:55:31.016875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:55:31.016875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:55:31.016875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:55:31.016875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:55:31.016875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:55:31.016875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:55:31.016875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:55:31.016875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:55:31.016875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:55:31.016875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:55:31.016875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 584/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:55:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:55:31.020847 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c65086-cc3c-41d6-881b-ced6f7b5a21c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:31.113210 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:31.135663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c65086-cc3c-41d6-881b-ced6f7b5a21c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:31.179877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffc574-cab3-4438-af8f-94d118643776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:31.180585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffc574-cab3-4438-af8f-94d118643776 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:31.180873 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffc574-cab3-4438-af8f-94d118643776 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:31.181147 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffc574-cab3-4438-af8f-94d118643776 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:31.181902 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ffc574-cab3-4438-af8f-94d118643776 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:55:31.182207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ffc574-cab3-4438-af8f-94d118643776 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:31.183569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ffc574-cab3-4438-af8f-94d118643776 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:31.192553 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 597/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:55:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:31.199260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa56a2-1626-400c-89cb-40d06094f5a1 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:31.299811 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa56a2-1626-400c-89cb-40d06094f5a1 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:31.336129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7b9f7-dceb-4324-a527-b12ad9ec41a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:31.337044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7b9f7-dceb-4324-a527-b12ad9ec41a6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:31.337318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7b9f7-dceb-4324-a527-b12ad9ec41a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:31.337589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7b9f7-dceb-4324-a527-b12ad9ec41a6 None admin] PATH_INFO: `/v3/users/e6a627db1fa642899a4299310c115edd` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:31.340367 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:31.351878 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7b9f7-dceb-4324-a527-b12ad9ec41a6 None admin] RBAC: Authorizing `identity:get_user(user_id=e6a627db1fa642899a4299310c115edd)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:31.352986 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c7b9f7-dceb-4324-a527-b12ad9ec41a6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:31.354682 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 585/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:55:31 2022] GET /identity/v3/users/e6a627db1fa642899a4299310c115edd => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:31.363693 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffc574-cab3-4438-af8f-94d118643776 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:31.475337 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffc574-cab3-4438-af8f-94d118643776 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:31.511882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d17efb-d629-4728-85a1-74444a9ddbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:31.512541 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d17efb-d629-4728-85a1-74444a9ddbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:31.512819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d17efb-d629-4728-85a1-74444a9ddbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:31.513090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d17efb-d629-4728-85a1-74444a9ddbf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:31.529381 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:31.584679 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d17efb-d629-4728-85a1-74444a9ddbf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:31.586100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d17efb-d629-4728-85a1-74444a9ddbf3 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:31.686036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 598/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:55:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:55:31.694906 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7b9f7-dceb-4324-a527-b12ad9ec41a6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:31.797911 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c7b9f7-dceb-4324-a527-b12ad9ec41a6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:31.839177 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d3b06-6270-4b5d-9e89-93880beb175c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:31.839976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d3b06-6270-4b5d-9e89-93880beb175c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:31.840259 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d3b06-6270-4b5d-9e89-93880beb175c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:31.840566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d3b06-6270-4b5d-9e89-93880beb175c None admin] PATH_INFO: `/v3/projects/475b8907d3fc4b0f867d2ff23cb81bc6` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:31.841834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d3b06-6270-4b5d-9e89-93880beb175c None admin] RBAC: Authorizing `identity:get_project(project_id=475b8907d3fc4b0f867d2ff23cb81bc6)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:31.843285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d3b06-6270-4b5d-9e89-93880beb175c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:31.845001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 586/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:55:31 2022] GET /identity/v3/projects/475b8907d3fc4b0f867d2ff23cb81bc6 => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:55:31.851492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d17efb-d629-4728-85a1-74444a9ddbf3 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:31.901423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:31.960845 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d17efb-d629-4728-85a1-74444a9ddbf3 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:31.995567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef6717-9f2a-4ad7-8456-df1a3eaf893d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:31.996204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef6717-9f2a-4ad7-8456-df1a3eaf893d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:31.996508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef6717-9f2a-4ad7-8456-df1a3eaf893d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:31.996778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef6717-9f2a-4ad7-8456-df1a3eaf893d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:31.997602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef6717-9f2a-4ad7-8456-df1a3eaf893d None admin] RBAC: Adding query filter params (role.id=b1e13c449fa64b3e88b6832bb767214b, scope.project.id=475b8907d3fc4b0f867d2ff23cb81bc6, user.id=e6a627db1fa642899a4299310c115edd) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:55:31.997909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef6717-9f2a-4ad7-8456-df1a3eaf893d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:31.999337 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef6717-9f2a-4ad7-8456-df1a3eaf893d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:32.004890 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 599/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:55:31 2022] GET /identity/v3/role_assignments?user.id=e6a627db1fa642899a4299310c115edd&scope.project.id=475b8907d3fc4b0f867d2ff23cb81bc6&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:33.981517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f624d6-65d5-4bfa-ae92-2582a33d8379 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:33.982060 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f624d6-65d5-4bfa-ae92-2582a33d8379 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:33.982060 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f624d6-65d5-4bfa-ae92-2582a33d8379 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:33.983445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 587/1186] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:55:33.989907 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dff0e-0d77-45d7-b6d7-14a657c71f98 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:33.990617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dff0e-0d77-45d7-b6d7-14a657c71f98 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:33.990777 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dff0e-0d77-45d7-b6d7-14a657c71f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:34.009971 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ca2dff0e-0d77-45d7-b6d7-14a657c71f98 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:55:34.011195 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ca2dff0e-0d77-45d7-b6d7-14a657c71f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:55:34.089662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 600/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:55:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:55:34.113812 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f624d6-65d5-4bfa-ae92-2582a33d8379 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:34.182060 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:34.224539 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f624d6-65d5-4bfa-ae92-2582a33d8379 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:34.261033 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103372f9-7891-4d38-b357-7b4d5c8976b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:34.261845 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103372f9-7891-4d38-b357-7b4d5c8976b9 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:34.262124 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103372f9-7891-4d38-b357-7b4d5c8976b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:34.262400 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103372f9-7891-4d38-b357-7b4d5c8976b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:34.274399 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103372f9-7891-4d38-b357-7b4d5c8976b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:55:34.282037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-103372f9-7891-4d38-b357-7b4d5c8976b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:55:34.282037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:55:34.282037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:55:34.282037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:55:34.282037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:55:34.282037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:55:34.282037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:55:34.282037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:55:34.282037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:55:34.282037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:55:34.282037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:55:34.282037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:55:34.282037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:55:34.282037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:55:34.282037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:55:34.282037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:55:34.283100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:55:34.283100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:55:34.283100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:55:34.283100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:55:34.283100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:55:34.283100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:55:34.283100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:55:34.283100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:55:34.283100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:55:34.283100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:55:34.283100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:55:34.283100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:55:34.283100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:55:34.283100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:55:34.283100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:55:34.283100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:55:34.284080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:55:34.284080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:55:34.284080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:55:34.284080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:55:34.284080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:55:34.284080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:55:34.284080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:55:34.284080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:55:34.284080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:55:34.284080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:55:34.284080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:55:34.284080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:55:34.288273 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 588/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:55:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:55:34.290019 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2dff0e-0d77-45d7-b6d7-14a657c71f98 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:34.332573 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:34.395479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2dff0e-0d77-45d7-b6d7-14a657c71f98 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:34.433215 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f3f51-2db7-476d-9acf-1d20bae4f3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:34.433454 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f3f51-2db7-476d-9acf-1d20bae4f3dd None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:34.433754 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f3f51-2db7-476d-9acf-1d20bae4f3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:34.434027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f3f51-2db7-476d-9acf-1d20bae4f3dd None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:34.434793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f3f51-2db7-476d-9acf-1d20bae4f3dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:55:34.435110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f3f51-2db7-476d-9acf-1d20bae4f3dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:34.436509 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f3f51-2db7-476d-9acf-1d20bae4f3dd None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:34.446238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 601/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:55:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:55:34.452805 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103372f9-7891-4d38-b357-7b4d5c8976b9 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:34.560640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103372f9-7891-4d38-b357-7b4d5c8976b9 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:34.583391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:34.595812 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b501d-6569-482e-baaa-eb93ff1a035d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:34.596710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b501d-6569-482e-baaa-eb93ff1a035d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:34.596990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b501d-6569-482e-baaa-eb93ff1a035d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:34.597260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b501d-6569-482e-baaa-eb93ff1a035d None admin] PATH_INFO: `/v3/users/e6a627db1fa642899a4299310c115edd` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:34.598459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b501d-6569-482e-baaa-eb93ff1a035d None admin] RBAC: Authorizing `identity:get_user(user_id=e6a627db1fa642899a4299310c115edd)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:34.599933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b501d-6569-482e-baaa-eb93ff1a035d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:34.601746 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 589/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:55:34 2022] GET /identity/v3/users/e6a627db1fa642899a4299310c115edd => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:34.610987 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f3f51-2db7-476d-9acf-1d20bae4f3dd None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:34.720441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f3f51-2db7-476d-9acf-1d20bae4f3dd None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:34.722036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:34.758131 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed903bb-9e0e-43a0-8e8b-bd9adff9bb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:34.758980 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed903bb-9e0e-43a0-8e8b-bd9adff9bb64 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:34.759255 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed903bb-9e0e-43a0-8e8b-bd9adff9bb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:34.759522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed903bb-9e0e-43a0-8e8b-bd9adff9bb64 None admin] PATH_INFO: `/v3/projects/475b8907d3fc4b0f867d2ff23cb81bc6` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:34.760779 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed903bb-9e0e-43a0-8e8b-bd9adff9bb64 None admin] RBAC: Authorizing `identity:get_project(project_id=475b8907d3fc4b0f867d2ff23cb81bc6)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:34.762262 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed903bb-9e0e-43a0-8e8b-bd9adff9bb64 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:34.763943 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 602/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:55:34 2022] GET /identity/v3/projects/475b8907d3fc4b0f867d2ff23cb81bc6 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:34.770288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b501d-6569-482e-baaa-eb93ff1a035d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:34.888516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b501d-6569-482e-baaa-eb93ff1a035d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:34.924120 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bf196-5e39-431a-b670-82076445838f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:34.925044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bf196-5e39-431a-b670-82076445838f None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:34.925317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bf196-5e39-431a-b670-82076445838f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:34.925585 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bf196-5e39-431a-b670-82076445838f None admin] PATH_INFO: `/v3/projects/475b8907d3fc4b0f867d2ff23cb81bc6/users/e6a627db1fa642899a4299310c115edd/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:34.927684 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bf196-5e39-431a-b670-82076445838f None admin] RBAC: Authorizing `identity:create_grant(project_id=475b8907d3fc4b0f867d2ff23cb81bc6, user_id=e6a627db1fa642899a4299310c115edd, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:34.929182 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668bf196-5e39-431a-b670-82076445838f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:34.944014 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 590/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:55:34 2022] PUT /identity/v3/projects/475b8907d3fc4b0f867d2ff23cb81bc6/users/e6a627db1fa642899a4299310c115edd/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:55:36.928247 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffac31-fc92-47c0-9583-850c428905ad None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:36.929163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffac31-fc92-47c0-9583-850c428905ad None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:36.929163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffac31-fc92-47c0-9583-850c428905ad None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:36.931171 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 603/1193] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:55:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:55:36.936680 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42f723-1096-400c-ae1f-a51c0986f95b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:36.937107 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42f723-1096-400c-ae1f-a51c0986f95b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:36.937215 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42f723-1096-400c-ae1f-a51c0986f95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:36.956625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5f42f723-1096-400c-ae1f-a51c0986f95b None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:55:36.957818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5f42f723-1096-400c-ae1f-a51c0986f95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:55:37.006641 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:37.029519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 591/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:55:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:55:37.063661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ffac31-fc92-47c0-9583-850c428905ad None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:37.170648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ffac31-fc92-47c0-9583-850c428905ad None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:37.206043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c69bb8-8173-461f-bf30-eaa43e2cc4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:37.206827 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c69bb8-8173-461f-bf30-eaa43e2cc4ab None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:37.207108 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c69bb8-8173-461f-bf30-eaa43e2cc4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:37.207379 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c69bb8-8173-461f-bf30-eaa43e2cc4ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:37.217912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c69bb8-8173-461f-bf30-eaa43e2cc4ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:55:37.225390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-47c69bb8-8173-461f-bf30-eaa43e2cc4ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:55:37.225390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:55:37.225390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:55:37.225390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:55:37.225390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:55:37.225390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:55:37.225390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:55:37.225390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:55:37.225390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:55:37.225390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:55:37.225390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:55:37.225390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:55:37.225390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:55:37.225390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:55:37.225390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:55:37.225390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:55:37.226519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:55:37.226519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:55:37.226519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:55:37.226519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:55:37.226519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:55:37.226519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:55:37.226519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:55:37.226519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:55:37.226519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:55:37.226519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:55:37.226519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:55:37.226519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:55:37.226519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:55:37.226519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:55:37.226519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:55:37.226519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:55:37.227522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:55:37.227522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:55:37.227522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:55:37.227522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:55:37.227522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:55:37.227522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:55:37.227522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:55:37.227522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:55:37.227522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:55:37.227522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:55:37.227522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:55:37.227522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:55:37.228595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 604/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:55:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:55:37.233025 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42f723-1096-400c-ae1f-a51c0986f95b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:37.354606 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42f723-1096-400c-ae1f-a51c0986f95b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:37.392174 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd55f4-d3f9-4183-93e3-6a99dde2f31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:37.392872 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd55f4-d3f9-4183-93e3-6a99dde2f31f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:37.393154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd55f4-d3f9-4183-93e3-6a99dde2f31f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:37.393427 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd55f4-d3f9-4183-93e3-6a99dde2f31f None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:37.394178 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd55f4-d3f9-4183-93e3-6a99dde2f31f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:55:37.394483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd55f4-d3f9-4183-93e3-6a99dde2f31f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:37.395864 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd55f4-d3f9-4183-93e3-6a99dde2f31f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:37.404764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 592/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:55:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:37.410513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c69bb8-8173-461f-bf30-eaa43e2cc4ab None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:37.445258 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:37.518512 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c69bb8-8173-461f-bf30-eaa43e2cc4ab None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:37.555707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931dd53a-0d8f-4fe5-aea5-8e88355582cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:37.556593 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dd53a-0d8f-4fe5-aea5-8e88355582cc None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:37.556875 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dd53a-0d8f-4fe5-aea5-8e88355582cc None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:37.557155 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dd53a-0d8f-4fe5-aea5-8e88355582cc None admin] PATH_INFO: `/v3/users/e6a627db1fa642899a4299310c115edd` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:37.558378 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931dd53a-0d8f-4fe5-aea5-8e88355582cc None admin] RBAC: Authorizing `identity:get_user(user_id=e6a627db1fa642899a4299310c115edd)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:37.559814 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931dd53a-0d8f-4fe5-aea5-8e88355582cc None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:37.561478 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 605/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Jun 25 15:55:37 2022] GET /identity/v3/users/e6a627db1fa642899a4299310c115edd => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:37.567930 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd55f4-d3f9-4183-93e3-6a99dde2f31f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:37.619847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:37.683630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd55f4-d3f9-4183-93e3-6a99dde2f31f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:37.732492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea07aa-208c-44e1-bb71-36d6ceb4c988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:37.733089 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea07aa-208c-44e1-bb71-36d6ceb4c988 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:37.733366 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea07aa-208c-44e1-bb71-36d6ceb4c988 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:37.733607 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea07aa-208c-44e1-bb71-36d6ceb4c988 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:37.803809 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea07aa-208c-44e1-bb71-36d6ceb4c988 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:37.805333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea07aa-208c-44e1-bb71-36d6ceb4c988 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:37.848674 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:37.910953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 593/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:55:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:55:37.918376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931dd53a-0d8f-4fe5-aea5-8e88355582cc None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:38.015965 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:38.020138 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931dd53a-0d8f-4fe5-aea5-8e88355582cc None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:38.056603 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ada5c-ae83-42ee-ab9e-02c2208f40d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:38.057460 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ada5c-ae83-42ee-ab9e-02c2208f40d7 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:38.057735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ada5c-ae83-42ee-ab9e-02c2208f40d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:38.058019 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ada5c-ae83-42ee-ab9e-02c2208f40d7 None admin] PATH_INFO: `/v3/projects/475b8907d3fc4b0f867d2ff23cb81bc6` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:38.059240 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ada5c-ae83-42ee-ab9e-02c2208f40d7 None admin] RBAC: Authorizing `identity:get_project(project_id=475b8907d3fc4b0f867d2ff23cb81bc6)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:38.060708 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ada5c-ae83-42ee-ab9e-02c2208f40d7 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:38.062406 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 606/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Jun 25 15:55:37 2022] GET /identity/v3/projects/475b8907d3fc4b0f867d2ff23cb81bc6 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:38.068712 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea07aa-208c-44e1-bb71-36d6ceb4c988 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:55:38.180051 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea07aa-208c-44e1-bb71-36d6ceb4c988 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:55:38.215628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9dea7-8abb-4676-9eee-30f58f5e7736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:55:38.216260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9dea7-8abb-4676-9eee-30f58f5e7736 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:55:38.216559 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9dea7-8abb-4676-9eee-30f58f5e7736 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:55:38.216829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9dea7-8abb-4676-9eee-30f58f5e7736 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:55:38.217653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9dea7-8abb-4676-9eee-30f58f5e7736 None admin] RBAC: Adding query filter params (role.id=b1e13c449fa64b3e88b6832bb767214b, scope.project.id=475b8907d3fc4b0f867d2ff23cb81bc6, user.id=e6a627db1fa642899a4299310c115edd) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:55:38.217965 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9dea7-8abb-4676-9eee-30f58f5e7736 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:55:38.219374 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9dea7-8abb-4676-9eee-30f58f5e7736 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:55:38.225336 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 594/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:55:38 2022] GET /identity/v3/role_assignments?user.id=e6a627db1fa642899a4299310c115edd&scope.project.id=475b8907d3fc4b0f867d2ff23cb81bc6&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:55:48.051783 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=17 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:55:48.222623 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:33.471035 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08137790-7a31-47e1-b1b4-fa659f3b830b None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:33.471645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08137790-7a31-47e1-b1b4-fa659f3b830b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:33.471754 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08137790-7a31-47e1-b1b4-fa659f3b830b None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:33.473401 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 607/1201] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:56:33.478977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56647e2-0a49-4454-a1d9-4c7b769036ea None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:33.479491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56647e2-0a49-4454-a1d9-4c7b769036ea None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:33.479582 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56647e2-0a49-4454-a1d9-4c7b769036ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:33.497653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f56647e2-0a49-4454-a1d9-4c7b769036ea None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:56:33.499070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f56647e2-0a49-4454-a1d9-4c7b769036ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:56:33.601276 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 595/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:56:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:56:33.631986 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08137790-7a31-47e1-b1b4-fa659f3b830b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:33.777639 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08137790-7a31-47e1-b1b4-fa659f3b830b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:33.819515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc108315-8aa2-4694-8266-bd1d1ce97d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:33.820506 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc108315-8aa2-4694-8266-bd1d1ce97d24 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:33.820794 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc108315-8aa2-4694-8266-bd1d1ce97d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:33.821072 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc108315-8aa2-4694-8266-bd1d1ce97d24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:33.834025 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc108315-8aa2-4694-8266-bd1d1ce97d24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:33.835687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc108315-8aa2-4694-8266-bd1d1ce97d24 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:33.860256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-bc108315-8aa2-4694-8266-bd1d1ce97d24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:33.860256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:33.860256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:33.860256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:33.860256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:33.860256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:33.860256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:33.860256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:33.860256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:33.860256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:33.860256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:33.860256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:33.860256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:56:33.860256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:56:33.860256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:56:33.861602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:56:33.861602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:33.861602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:33.861602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:33.861602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:33.861602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:33.861602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:33.861602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:33.861602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:33.861602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:33.861602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:33.861602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:33.861602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:33.861602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:33.861602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:33.861602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:33.861602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:33.862757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:56:33.862757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:56:33.862757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:33.862757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:56:33.868458 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 608/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:56:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:33.868629 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56647e2-0a49-4454-a1d9-4c7b769036ea None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:33.995831 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56647e2-0a49-4454-a1d9-4c7b769036ea None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:34.029521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:34.036853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000818fb-c7e5-4f28-9807-a9c9e56e5130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:34.037678 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000818fb-c7e5-4f28-9807-a9c9e56e5130 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:34.037995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000818fb-c7e5-4f28-9807-a9c9e56e5130 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:34.038403 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000818fb-c7e5-4f28-9807-a9c9e56e5130 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:34.039376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000818fb-c7e5-4f28-9807-a9c9e56e5130 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:34.039872 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000818fb-c7e5-4f28-9807-a9c9e56e5130 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:34.042859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000818fb-c7e5-4f28-9807-a9c9e56e5130 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:34.060453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 596/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:34.063435 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc108315-8aa2-4694-8266-bd1d1ce97d24 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:34.143813 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:34.203032 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc108315-8aa2-4694-8266-bd1d1ce97d24 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:34.245623 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2926f-0afd-4b44-8859-49747741dc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:34.246645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2926f-0afd-4b44-8859-49747741dc17 None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:34.247120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2926f-0afd-4b44-8859-49747741dc17 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:34.247612 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2926f-0afd-4b44-8859-49747741dc17 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:34.248330 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2926f-0afd-4b44-8859-49747741dc17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:34.249808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2926f-0afd-4b44-8859-49747741dc17 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:34.272450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 609/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Jun 25 15:56:34 2022] POST /identity/v3/users => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 15:56:36.444586 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3df45f-b3fb-4bec-bf72-bce1482f724c None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:36.445120 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3df45f-b3fb-4bec-bf72-bce1482f724c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:36.445120 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3df45f-b3fb-4bec-bf72-bce1482f724c None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:36.446781 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 597/1206] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:56:36.452546 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612862a-a9c6-4fbb-8725-4429ba9791f5 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:36.453056 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612862a-a9c6-4fbb-8725-4429ba9791f5 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:36.453156 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612862a-a9c6-4fbb-8725-4429ba9791f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:36.472540 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-d612862a-a9c6-4fbb-8725-4429ba9791f5 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:56:36.473716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d612862a-a9c6-4fbb-8725-4429ba9791f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:56:36.554498 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 610/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:56:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:56:36.579161 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3df45f-b3fb-4bec-bf72-bce1482f724c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:36.702324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3df45f-b3fb-4bec-bf72-bce1482f724c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:36.738074 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe8750-6f52-4ce2-beae-d179b5ebcc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:36.738933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe8750-6f52-4ce2-beae-d179b5ebcc64 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:36.739201 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe8750-6f52-4ce2-beae-d179b5ebcc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:36.739480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe8750-6f52-4ce2-beae-d179b5ebcc64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:36.751920 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe8750-6f52-4ce2-beae-d179b5ebcc64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:56:36.759889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-f9fe8750-6f52-4ce2-beae-d179b5ebcc64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:56:36.759889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:36.759889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:36.759889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:36.759889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:36.759889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:36.759889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:36.759889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:36.759889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:36.759889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:36.759889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:36.759889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:36.759889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:56:36.759889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:56:36.759889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:56:36.759889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:56:36.761368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:56:36.761368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:56:36.761368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:56:36.761368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:56:36.761368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:36.761368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:36.761368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:36.761368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:36.761368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:36.761368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:36.761368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:36.761368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:36.761368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:36.761368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:36.761368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:36.761368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:36.762341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:36.762341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:36.762341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:36.762341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:36.762341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:56:36.762341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:56:36.762341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:56:36.762341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:56:36.762341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:56:36.762341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:56:36.762341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:56:36.762341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:56:36.762341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 598/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:56:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:36.767863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612862a-a9c6-4fbb-8725-4429ba9791f5 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:36.815163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:36.893961 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612862a-a9c6-4fbb-8725-4429ba9791f5 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:36.933038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af0dc3-b820-4417-9382-b212c450fd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:36.933708 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af0dc3-b820-4417-9382-b212c450fd4c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:36.933984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af0dc3-b820-4417-9382-b212c450fd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:36.934295 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af0dc3-b820-4417-9382-b212c450fd4c None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:36.935051 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af0dc3-b820-4417-9382-b212c450fd4c None admin] RBAC: Adding query filter params (name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:36.935360 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af0dc3-b820-4417-9382-b212c450fd4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:36.936892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8af0dc3-b820-4417-9382-b212c450fd4c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:36.945823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 611/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:56:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:36.954033 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe8750-6f52-4ce2-beae-d179b5ebcc64 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:37.012277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:37.068260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe8750-6f52-4ce2-beae-d179b5ebcc64 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:37.104929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4661190-ac9f-4998-b6df-bd7f4edc15a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:37.105825 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4661190-ac9f-4998-b6df-bd7f4edc15a0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:37.106118 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4661190-ac9f-4998-b6df-bd7f4edc15a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:37.106396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4661190-ac9f-4998-b6df-bd7f4edc15a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:37.117442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4661190-ac9f-4998-b6df-bd7f4edc15a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:37.118956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4661190-ac9f-4998-b6df-bd7f4edc15a0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:37.142390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-d4661190-ac9f-4998-b6df-bd7f4edc15a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:37.142390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:37.142390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:37.142390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:37.142390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:37.142390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:37.142390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:37.142390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:37.142390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:37.142390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:37.142390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:37.142390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:37.142390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:56:37.142390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:56:37.142390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:56:37.143737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:56:37.143737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:37.143737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:37.143737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:37.143737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:37.143737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:37.143737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:37.143737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:37.143737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:37.143737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:37.143737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:37.143737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:37.143737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:37.143737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:37.143737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:37.143737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:37.143737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:37.144732 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:56:37.144732 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:56:37.144732 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:37.144732 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:56:37.144732 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 599/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:56:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:56:37.150004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af0dc3-b820-4417-9382-b212c450fd4c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:37.264755 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:37.268672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af0dc3-b820-4417-9382-b212c450fd4c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:37.308709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3332c-fa0a-4b95-bd3d-eca64b09f07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:37.309429 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3332c-fa0a-4b95-bd3d-eca64b09f07c None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:37.309713 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3332c-fa0a-4b95-bd3d-eca64b09f07c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:37.310028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3332c-fa0a-4b95-bd3d-eca64b09f07c None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:37.310940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3332c-fa0a-4b95-bd3d-eca64b09f07c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:37.311319 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3332c-fa0a-4b95-bd3d-eca64b09f07c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:37.312750 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3332c-fa0a-4b95-bd3d-eca64b09f07c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:37.326999 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 612/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:56:37.333563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4661190-ac9f-4998-b6df-bd7f4edc15a0 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:37.456950 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4661190-ac9f-4998-b6df-bd7f4edc15a0 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:37.459080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:37.498197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5169ed3-a532-4c5d-84f4-a92b1395d73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:37.499044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5169ed3-a532-4c5d-84f4-a92b1395d73d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:37.499380 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5169ed3-a532-4c5d-84f4-a92b1395d73d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:37.499660 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5169ed3-a532-4c5d-84f4-a92b1395d73d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:37.508597 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5169ed3-a532-4c5d-84f4-a92b1395d73d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:37.510645 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5169ed3-a532-4c5d-84f4-a92b1395d73d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:37.531639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-e5169ed3-a532-4c5d-84f4-a92b1395d73d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 25 15:56:37.531639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:37.531639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:37.531639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:37.531639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:37.531639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:37.531639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:37.531639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:37.531639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:37.531639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:37.531639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:37.531639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:37.531639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:56:37.531639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:56:37.531639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:56:37.531639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:56:37.532951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:37.532951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:37.532951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:56:37.532951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:56:37.532951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:56:37.532951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:56:37.532951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:37.532951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:37.532951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:37.532951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:37.532951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:37.532951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:37.532951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:37.532951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:37.532951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:37.532951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:37.532951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:37.532951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:37.533940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:37.533940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:37.533940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:56:37.533940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:56:37.533940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:56:37.533940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:56:37.533940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:56:37.533940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:56:37.533940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 25 15:56:37.533940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:56:37.533940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 600/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:56:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:56:37.535727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3332c-fa0a-4b95-bd3d-eca64b09f07c None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:37.663351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3332c-fa0a-4b95-bd3d-eca64b09f07c None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:37.703161 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6f433-36f1-4bb2-a5fc-8710bf20c7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:37.703868 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6f433-36f1-4bb2-a5fc-8710bf20c7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:37.704143 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6f433-36f1-4bb2-a5fc-8710bf20c7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:37.704471 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6f433-36f1-4bb2-a5fc-8710bf20c7d8 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:37.705343 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec6f433-36f1-4bb2-a5fc-8710bf20c7d8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:37.705650 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec6f433-36f1-4bb2-a5fc-8710bf20c7d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:37.707118 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec6f433-36f1-4bb2-a5fc-8710bf20c7d8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:37.730516 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 613/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Sat Jun 25 15:56:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:37.731783 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5169ed3-a532-4c5d-84f4-a92b1395d73d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:37.855141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5169ed3-a532-4c5d-84f4-a92b1395d73d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:37.894680 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ea715-ad3a-4e2c-808f-14926710ad3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:37.895396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ea715-ad3a-4e2c-808f-14926710ad3c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:37.895734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ea715-ad3a-4e2c-808f-14926710ad3c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:37.896036 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ea715-ad3a-4e2c-808f-14926710ad3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:37.930900 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:37.978557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ea715-ad3a-4e2c-808f-14926710ad3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:37.980234 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ea715-ad3a-4e2c-808f-14926710ad3c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:38.093557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 601/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:56:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:56:38.099366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6f433-36f1-4bb2-a5fc-8710bf20c7d8 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:38.120798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:38.216918 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6f433-36f1-4bb2-a5fc-8710bf20c7d8 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:38.262827 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba5be5-2215-4dd6-8562-e528d77b8ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:38.263886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba5be5-2215-4dd6-8562-e528d77b8ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:38.264244 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba5be5-2215-4dd6-8562-e528d77b8ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:38.264636 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba5be5-2215-4dd6-8562-e528d77b8ce4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:38.276782 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba5be5-2215-4dd6-8562-e528d77b8ce4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:38.278376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba5be5-2215-4dd6-8562-e528d77b8ce4 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:38.297920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-d3ba5be5-2215-4dd6-8562-e528d77b8ce4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:38.297920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:38.297920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:38.297920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:38.297920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:38.297920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:38.297920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:38.297920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:38.297920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:38.297920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:38.297920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:38.297920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:38.297920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:56:38.297920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:56:38.297920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:56:38.298949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:56:38.298949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:38.298949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:38.298949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:38.298949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:38.298949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:38.298949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:38.298949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:38.298949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:38.298949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:38.298949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:38.298949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:38.298949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:38.298949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:38.298949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:38.298949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:38.298949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:38.301529 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:56:38.301529 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:56:38.301529 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:38.301529 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:56:38.301529 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 614/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:56:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:56:38.305081 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ea715-ad3a-4e2c-808f-14926710ad3c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:38.384303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:38.433920 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ea715-ad3a-4e2c-808f-14926710ad3c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:38.471488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b594758-34e9-4e69-a945-4550b6594417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:38.472186 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b594758-34e9-4e69-a945-4550b6594417 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:38.472469 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b594758-34e9-4e69-a945-4550b6594417 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:38.472705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b594758-34e9-4e69-a945-4550b6594417 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:38.473436 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b594758-34e9-4e69-a945-4550b6594417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:38.473741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b594758-34e9-4e69-a945-4550b6594417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:38.475131 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b594758-34e9-4e69-a945-4550b6594417 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:38.487009 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 602/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:38.492726 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba5be5-2215-4dd6-8562-e528d77b8ce4 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:38.574711 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:38.616079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba5be5-2215-4dd6-8562-e528d77b8ce4 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:38.666510 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee69ee-37ac-4997-b9b7-af454b4f43ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:38.667314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee69ee-37ac-4997-b9b7-af454b4f43ed None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:38.667595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee69ee-37ac-4997-b9b7-af454b4f43ed None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:38.667869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee69ee-37ac-4997-b9b7-af454b4f43ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:38.677354 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee69ee-37ac-4997-b9b7-af454b4f43ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:38.678814 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee69ee-37ac-4997-b9b7-af454b4f43ed None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:38.693242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-82ee69ee-37ac-4997-b9b7-af454b4f43ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:56:38.693242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:38.693242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:38.693242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:38.693242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:38.693242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:38.693242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:38.693242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:38.693242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:38.693242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:38.693242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:38.693242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:38.693242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:56:38.693242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:56:38.693242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:56:38.694276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:56:38.694276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:38.694276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:38.694276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:38.694276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:38.694276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:38.694276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:38.694276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:38.694276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:38.694276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:38.694276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:38.694276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:38.694276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:38.694276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:38.694276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:38.694276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:38.694276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:38.695170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:56:38.695170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:56:38.695170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:56:38.695170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:56:38.695170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:56:38.695170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:56:38.695170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:56:38.695170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:56:38.695170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 615/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:56:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:56:38.701307 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b594758-34e9-4e69-a945-4550b6594417 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:38.814408 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b594758-34e9-4e69-a945-4550b6594417 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:38.838196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:38.862294 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1811c-7bb4-4e6e-9213-4ffdd3abfda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:38.863396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1811c-7bb4-4e6e-9213-4ffdd3abfda1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:38.863968 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1811c-7bb4-4e6e-9213-4ffdd3abfda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:38.864498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1811c-7bb4-4e6e-9213-4ffdd3abfda1 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:38.865552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1811c-7bb4-4e6e-9213-4ffdd3abfda1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:38.866161 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1811c-7bb4-4e6e-9213-4ffdd3abfda1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:38.868054 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac1811c-7bb4-4e6e-9213-4ffdd3abfda1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:38.884567 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 603/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:56:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:38.890827 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee69ee-37ac-4997-b9b7-af454b4f43ed None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:39.008649 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee69ee-37ac-4997-b9b7-af454b4f43ed None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:39.041948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:39.048771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b4503-ace5-4d1b-a710-1b857b75d507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:39.049526 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b4503-ace5-4d1b-a710-1b857b75d507 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:39.049828 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b4503-ace5-4d1b-a710-1b857b75d507 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:39.050453 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b4503-ace5-4d1b-a710-1b857b75d507 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:39.051664 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b4503-ace5-4d1b-a710-1b857b75d507 None admin] RBAC: Adding query filter params (role.id=7f1996bcf7e54638b4f53b3d8ba83495, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=474c468a610140f8a5b18d5a1ed1d18e) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:39.052281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b4503-ace5-4d1b-a710-1b857b75d507 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:39.053894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b4503-ace5-4d1b-a710-1b857b75d507 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:39.059900 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 616/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:56:38 2022] GET /identity/v3/role_assignments?user.id=474c468a610140f8a5b18d5a1ed1d18e&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=7f1996bcf7e54638b4f53b3d8ba83495 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:41.168856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133ee7f-51a0-4019-b3f6-675a43a304cf None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:41.169392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133ee7f-51a0-4019-b3f6-675a43a304cf None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:41.169522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133ee7f-51a0-4019-b3f6-675a43a304cf None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:41.170999 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 604/1220] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:56:41.177010 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30ccf6-299f-4297-b8cc-7c70b3103505 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:41.177587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30ccf6-299f-4297-b8cc-7c70b3103505 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:41.177781 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30ccf6-299f-4297-b8cc-7c70b3103505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:41.195482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ae30ccf6-299f-4297-b8cc-7c70b3103505 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:56:41.196858 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ae30ccf6-299f-4297-b8cc-7c70b3103505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:56:41.273012 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 617/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:56:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:56:41.298027 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133ee7f-51a0-4019-b3f6-675a43a304cf None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:41.415716 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133ee7f-51a0-4019-b3f6-675a43a304cf None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:41.458106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ce842-a32b-4cfa-a203-7983b696cfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:41.459115 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ce842-a32b-4cfa-a203-7983b696cfd6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:41.459493 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ce842-a32b-4cfa-a203-7983b696cfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:41.459674 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ce842-a32b-4cfa-a203-7983b696cfd6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:41.472572 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ce842-a32b-4cfa-a203-7983b696cfd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:56:41.482354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-743ce842-a32b-4cfa-a203-7983b696cfd6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:56:41.482354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:41.482354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:41.482354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:41.482354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:41.482354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:41.482354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:41.482354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:41.482354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:41.482354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:41.482354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:41.482354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:41.482354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:56:41.482354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:56:41.482354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:56:41.482354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:56:41.483551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:56:41.483551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:56:41.483551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:56:41.483551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:56:41.483551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:41.483551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:41.483551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:41.483551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:41.483551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:41.483551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:41.483551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:41.483551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:41.483551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:41.483551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:41.483551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:41.483551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:41.484715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:41.484715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:41.484715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:41.484715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:41.484715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:56:41.484715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:56:41.484715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:56:41.484715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:56:41.484715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:56:41.484715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:56:41.484715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:56:41.484715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:56:41.484715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 605/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:56:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:41.490988 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30ccf6-299f-4297-b8cc-7c70b3103505 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:41.602968 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30ccf6-299f-4297-b8cc-7c70b3103505 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:41.632146 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:41.646994 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86703a5d-6d67-4a1e-ab9e-24711501da77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:41.648026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86703a5d-6d67-4a1e-ab9e-24711501da77 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:41.648445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86703a5d-6d67-4a1e-ab9e-24711501da77 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:41.648731 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86703a5d-6d67-4a1e-ab9e-24711501da77 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:41.649706 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86703a5d-6d67-4a1e-ab9e-24711501da77 None admin] RBAC: Adding query filter params (name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:41.649984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86703a5d-6d67-4a1e-ab9e-24711501da77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:41.651696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86703a5d-6d67-4a1e-ab9e-24711501da77 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:41.661046 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 618/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:56:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:41.667253 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ce842-a32b-4cfa-a203-7983b696cfd6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:41.715521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:41.778173 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ce842-a32b-4cfa-a203-7983b696cfd6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:41.811840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881628a-c6fd-4f36-a7c3-e805187580e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:41.812628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881628a-c6fd-4f36-a7c3-e805187580e4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:41.812906 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881628a-c6fd-4f36-a7c3-e805187580e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:41.813171 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881628a-c6fd-4f36-a7c3-e805187580e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:41.828008 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881628a-c6fd-4f36-a7c3-e805187580e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:41.829734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881628a-c6fd-4f36-a7c3-e805187580e4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:41.850069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-f881628a-c6fd-4f36-a7c3-e805187580e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:41.850069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:41.850069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:41.850069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:41.850069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:41.850069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:41.850069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:41.850069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:41.850069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:41.850069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:41.850069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:41.850069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:41.850069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:56:41.850069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:56:41.850069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:56:41.851345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:56:41.851345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:41.851345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:41.851345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:41.851345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:41.851345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:41.851345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:41.851345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:41.851345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:41.851345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:41.851345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:41.851345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:41.851345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:41.851345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:41.851345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:41.851345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:41.851345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:41.852246 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:56:41.852246 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:56:41.852246 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:41.852246 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:56:41.852246 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 606/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:56:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:41.857524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86703a5d-6d67-4a1e-ab9e-24711501da77 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:41.966698 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86703a5d-6d67-4a1e-ab9e-24711501da77 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:42.000561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07ceb9-396e-463c-abeb-b706ca47b058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:42.001251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07ceb9-396e-463c-abeb-b706ca47b058 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:42.001529 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07ceb9-396e-463c-abeb-b706ca47b058 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:42.001797 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07ceb9-396e-463c-abeb-b706ca47b058 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:42.002514 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07ceb9-396e-463c-abeb-b706ca47b058 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:42.002837 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07ceb9-396e-463c-abeb-b706ca47b058 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:42.004187 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07ceb9-396e-463c-abeb-b706ca47b058 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:42.017536 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 619/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:42.023668 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881628a-c6fd-4f36-a7c3-e805187580e4 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:42.119799 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:42.126603 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881628a-c6fd-4f36-a7c3-e805187580e4 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:42.167373 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec8b92-0f77-477b-9aed-a8fef7343bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:42.168241 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec8b92-0f77-477b-9aed-a8fef7343bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:42.168567 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec8b92-0f77-477b-9aed-a8fef7343bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:42.168921 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec8b92-0f77-477b-9aed-a8fef7343bb3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:42.177543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec8b92-0f77-477b-9aed-a8fef7343bb3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:42.179010 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec8b92-0f77-477b-9aed-a8fef7343bb3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:42.193706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-ebec8b92-0f77-477b-9aed-a8fef7343bb3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 25 15:56:42.193706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:42.193706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:42.193706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:42.193706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:42.193706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:42.193706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:42.193706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:42.193706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:42.193706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:42.193706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:42.193706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:42.193706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:56:42.193706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:56:42.193706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:56:42.193706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:56:42.194770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:42.194770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:42.194770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:56:42.194770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:56:42.194770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:56:42.194770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:56:42.194770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:42.194770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:42.194770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:42.194770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:42.194770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:42.194770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:42.194770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:42.194770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:42.194770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:42.194770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:42.194770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:42.194770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:42.201088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:42.201088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:42.201088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:56:42.201088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:56:42.201088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:56:42.201088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:56:42.201088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:56:42.201088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:56:42.201088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 25 15:56:42.201088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:56:42.201088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 607/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:56:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:42.201837 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07ceb9-396e-463c-abeb-b706ca47b058 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:42.250117 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:42.305563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07ceb9-396e-463c-abeb-b706ca47b058 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:42.341716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d710eb5-ad40-494b-83bf-06f81f7047b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:42.342376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d710eb5-ad40-494b-83bf-06f81f7047b1 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:42.342692 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d710eb5-ad40-494b-83bf-06f81f7047b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:42.343023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d710eb5-ad40-494b-83bf-06f81f7047b1 None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:42.343950 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d710eb5-ad40-494b-83bf-06f81f7047b1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:42.344276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d710eb5-ad40-494b-83bf-06f81f7047b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:42.345862 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d710eb5-ad40-494b-83bf-06f81f7047b1 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:42.365408 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 620/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Sat Jun 25 15:56:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:42.371309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec8b92-0f77-477b-9aed-a8fef7343bb3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:42.477887 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec8b92-0f77-477b-9aed-a8fef7343bb3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:42.514991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5c06f-e8b8-4fe8-97cd-39062b3db472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:42.515705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5c06f-e8b8-4fe8-97cd-39062b3db472 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:42.516310 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5c06f-e8b8-4fe8-97cd-39062b3db472 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:42.516615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5c06f-e8b8-4fe8-97cd-39062b3db472 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:42.527577 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5c06f-e8b8-4fe8-97cd-39062b3db472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:42.529214 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5c06f-e8b8-4fe8-97cd-39062b3db472 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:42.549946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-78e5c06f-e8b8-4fe8-97cd-39062b3db472 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:42.549946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:42.549946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:42.549946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:42.549946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:42.549946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:42.549946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:42.549946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:42.549946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:42.549946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:42.549946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:42.549946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:42.549946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:56:42.549946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:56:42.549946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:56:42.551138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:56:42.551138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:42.551138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:42.551138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:42.551138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:42.551138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:42.551138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:42.551138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:42.551138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:42.551138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:42.551138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:42.551138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:42.551138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:42.551138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:42.551138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:42.551138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:42.551138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:42.552044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:56:42.552044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:56:42.552044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:42.552044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:56:42.552044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 608/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:56:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:42.557553 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d710eb5-ad40-494b-83bf-06f81f7047b1 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:42.685692 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d710eb5-ad40-494b-83bf-06f81f7047b1 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:42.688643 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:42.724125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03606c70-f6c7-4da1-869a-cd8c2aa18dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:42.724879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03606c70-f6c7-4da1-869a-cd8c2aa18dfc None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:42.725251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03606c70-f6c7-4da1-869a-cd8c2aa18dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:42.725413 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03606c70-f6c7-4da1-869a-cd8c2aa18dfc None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:42.726135 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03606c70-f6c7-4da1-869a-cd8c2aa18dfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:42.726444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03606c70-f6c7-4da1-869a-cd8c2aa18dfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:42.728807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03606c70-f6c7-4da1-869a-cd8c2aa18dfc None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:42.741213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 621/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:42.747545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5c06f-e8b8-4fe8-97cd-39062b3db472 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:42.757478 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:42.858604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5c06f-e8b8-4fe8-97cd-39062b3db472 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:42.903295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c83e7b-109d-4ede-a76a-f027fd6467b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:42.904302 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c83e7b-109d-4ede-a76a-f027fd6467b5 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:42.904612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c83e7b-109d-4ede-a76a-f027fd6467b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:42.905032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c83e7b-109d-4ede-a76a-f027fd6467b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:42.912042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c83e7b-109d-4ede-a76a-f027fd6467b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:42.913943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c83e7b-109d-4ede-a76a-f027fd6467b5 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:42.933516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-c8c83e7b-109d-4ede-a76a-f027fd6467b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:56:42.933516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:42.933516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:42.933516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:42.933516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:42.933516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:42.933516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:42.933516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:42.933516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:42.933516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:42.933516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:42.933516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:42.933516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:56:42.933516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:56:42.933516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:56:42.934549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:56:42.934549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:42.934549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:42.934549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:42.934549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:42.934549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:42.934549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:42.934549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:42.934549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:42.934549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:42.934549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:42.934549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:42.934549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:42.934549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:42.934549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:42.934549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:42.934549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:42.935855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:56:42.935855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:56:42.935855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:56:42.935855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:56:42.935855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:56:42.935855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:56:42.935855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:56:42.935855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:56:42.935855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 609/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:56:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:42.941601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03606c70-f6c7-4da1-869a-cd8c2aa18dfc None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:43.048174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03606c70-f6c7-4da1-869a-cd8c2aa18dfc None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:43.087000 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db8c63-1f97-4ae0-a2bd-1bfbfde9163d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:43.087727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db8c63-1f97-4ae0-a2bd-1bfbfde9163d None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:43.088007 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db8c63-1f97-4ae0-a2bd-1bfbfde9163d None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:43.088278 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db8c63-1f97-4ae0-a2bd-1bfbfde9163d None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:43.089102 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db8c63-1f97-4ae0-a2bd-1bfbfde9163d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:43.089431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db8c63-1f97-4ae0-a2bd-1bfbfde9163d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:43.090899 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db8c63-1f97-4ae0-a2bd-1bfbfde9163d None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:43.103825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 622/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:56:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:43.110045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c83e7b-109d-4ede-a76a-f027fd6467b5 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:43.185319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:43.225606 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c83e7b-109d-4ede-a76a-f027fd6467b5 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:43.263877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4784c3-158a-45ef-8a7e-48ccbd80efb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:43.264936 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4784c3-158a-45ef-8a7e-48ccbd80efb2 None admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:43.265190 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4784c3-158a-45ef-8a7e-48ccbd80efb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:43.265465 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4784c3-158a-45ef-8a7e-48ccbd80efb2 None admin] PATH_INFO: `/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/474c468a610140f8a5b18d5a1ed1d18e/roles/7f1996bcf7e54638b4f53b3d8ba83495` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:43.282293 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4784c3-158a-45ef-8a7e-48ccbd80efb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3600648eb94b434b8c797d2f454e46b8, user_id=474c468a610140f8a5b18d5a1ed1d18e, role_id=7f1996bcf7e54638b4f53b3d8ba83495)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:43.283851 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4784c3-158a-45ef-8a7e-48ccbd80efb2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:43.298445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 610/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:56:43 2022] PUT /identity/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/474c468a610140f8a5b18d5a1ed1d18e/roles/7f1996bcf7e54638b4f53b3d8ba83495 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:56:45.468288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d94c0b-c512-493c-8459-820f0de61cda None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:45.468288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d94c0b-c512-493c-8459-820f0de61cda None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:45.468288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d94c0b-c512-493c-8459-820f0de61cda None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:45.468288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 623/1233] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:56:45.474405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbb8a2-70a2-4477-8dfa-045ee6065acd None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:45.474901 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbb8a2-70a2-4477-8dfa-045ee6065acd None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:45.475032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbb8a2-70a2-4477-8dfa-045ee6065acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:45.491660 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-8afbb8a2-70a2-4477-8dfa-045ee6065acd None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:56:45.492910 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8afbb8a2-70a2-4477-8dfa-045ee6065acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:56:45.577459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 611/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:56:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:56:45.604372 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d94c0b-c512-493c-8459-820f0de61cda None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:45.636467 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:45.721687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d94c0b-c512-493c-8459-820f0de61cda None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:45.759079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb61d9-8ea9-44c8-81f2-7513ee45230f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:45.759879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb61d9-8ea9-44c8-81f2-7513ee45230f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:45.760155 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb61d9-8ea9-44c8-81f2-7513ee45230f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:45.760430 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb61d9-8ea9-44c8-81f2-7513ee45230f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:45.770884 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb61d9-8ea9-44c8-81f2-7513ee45230f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:56:45.778686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-66cb61d9-8ea9-44c8-81f2-7513ee45230f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:56:45.778686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:45.778686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:45.778686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:45.778686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:45.778686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:45.778686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:45.778686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:45.778686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:45.778686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:45.778686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:45.778686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:45.778686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:56:45.778686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:56:45.778686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:56:45.778686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:56:45.779822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:56:45.779822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:56:45.779822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:56:45.779822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:56:45.779822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:45.779822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:45.779822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:45.779822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:45.779822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:45.779822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:45.779822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:45.779822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:45.779822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:45.779822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:45.779822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:45.779822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:45.780725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:45.780725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:45.780725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:45.780725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:45.780725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:56:45.780725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:56:45.780725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:56:45.780725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:56:45.780725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:56:45.780725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:56:45.780725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 25 15:56:45.780725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:56:45.780725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 624/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Jun 25 15:56:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:56:45.786316 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbb8a2-70a2-4477-8dfa-045ee6065acd None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:45.833855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:45.891777 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbb8a2-70a2-4477-8dfa-045ee6065acd None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:45.927927 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6ce91-6b3c-49c5-80bc-4e4f0a5047b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:45.928675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6ce91-6b3c-49c5-80bc-4e4f0a5047b0 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:45.928958 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6ce91-6b3c-49c5-80bc-4e4f0a5047b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:45.929252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6ce91-6b3c-49c5-80bc-4e4f0a5047b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:45.929996 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6ce91-6b3c-49c5-80bc-4e4f0a5047b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:45.930381 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6ce91-6b3c-49c5-80bc-4e4f0a5047b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:45.931889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6ce91-6b3c-49c5-80bc-4e4f0a5047b0 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:45.941121 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 612/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Jun 25 15:56:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:45.947548 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb61d9-8ea9-44c8-81f2-7513ee45230f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:46.036321 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:46.053881 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb61d9-8ea9-44c8-81f2-7513ee45230f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:46.088780 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765fed2-39e3-4f6c-bc72-7167dbd6b5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:46.089776 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765fed2-39e3-4f6c-bc72-7167dbd6b5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:46.090204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765fed2-39e3-4f6c-bc72-7167dbd6b5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:46.090589 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765fed2-39e3-4f6c-bc72-7167dbd6b5d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:46.102084 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765fed2-39e3-4f6c-bc72-7167dbd6b5d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:46.103978 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765fed2-39e3-4f6c-bc72-7167dbd6b5d4 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:46.122972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-4765fed2-39e3-4f6c-bc72-7167dbd6b5d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:46.122972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:46.122972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:46.122972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:46.122972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:46.122972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:46.122972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:46.122972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:46.122972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:46.122972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:46.122972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:46.122972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:46.122972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:56:46.122972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:56:46.122972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:56:46.124124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:56:46.124124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:46.124124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:46.124124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:46.124124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:46.124124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:46.124124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:46.124124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:46.124124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:46.124124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:46.124124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:46.124124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:46.124124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:46.124124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:46.124124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:46.124124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:46.124124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:46.125250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:56:46.125250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:56:46.125250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:46.125250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:56:46.125250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 625/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:56:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:46.130112 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6ce91-6b3c-49c5-80bc-4e4f0a5047b0 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:46.235424 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6ce91-6b3c-49c5-80bc-4e4f0a5047b0 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:46.245833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:46.272294 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ea4e5-5266-4ea2-a362-3f50795ebd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:46.273093 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ea4e5-5266-4ea2-a362-3f50795ebd3c None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:46.273377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ea4e5-5266-4ea2-a362-3f50795ebd3c None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:46.273693 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ea4e5-5266-4ea2-a362-3f50795ebd3c None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:46.274535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ea4e5-5266-4ea2-a362-3f50795ebd3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:46.274893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ea4e5-5266-4ea2-a362-3f50795ebd3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:46.276505 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ea4e5-5266-4ea2-a362-3f50795ebd3c None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:46.294582 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 613/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:46.295772 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765fed2-39e3-4f6c-bc72-7167dbd6b5d4 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:46.405528 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765fed2-39e3-4f6c-bc72-7167dbd6b5d4 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:46.443890 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ff9e3-96f0-4279-9bca-891dd47ddafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:46.444796 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ff9e3-96f0-4279-9bca-891dd47ddafc None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:46.445084 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ff9e3-96f0-4279-9bca-891dd47ddafc None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:46.445358 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ff9e3-96f0-4279-9bca-891dd47ddafc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:46.448566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:46.454509 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ff9e3-96f0-4279-9bca-891dd47ddafc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:46.456127 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ff9e3-96f0-4279-9bca-891dd47ddafc None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:46.471281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-8a2ff9e3-96f0-4279-9bca-891dd47ddafc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 25 15:56:46.471281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:46.471281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:46.471281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:46.471281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:46.471281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:46.471281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:46.471281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:46.471281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:46.471281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:46.471281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:46.471281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:46.471281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:56:46.471281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:56:46.471281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:56:46.471281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:56:46.477620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:46.477620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:46.477620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:56:46.477620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:56:46.477620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:56:46.477620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:56:46.477620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:46.477620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:46.477620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:46.477620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:46.477620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:46.477620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:46.477620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:46.477620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:46.477620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:46.477620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:46.477620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:46.477620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:46.478791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:46.478791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:46.478791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:56:46.478791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:56:46.478791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:56:46.478791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:56:46.478791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:56:46.478791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:56:46.478791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 25 15:56:46.478791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:56:46.478791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 626/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:56:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:46.479467 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ea4e5-5266-4ea2-a362-3f50795ebd3c None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:46.586438 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ea4e5-5266-4ea2-a362-3f50795ebd3c None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:46.627237 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d47b2-cdd5-4fac-8ed3-8076f2e8dfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:46.628032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d47b2-cdd5-4fac-8ed3-8076f2e8dfe6 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:46.628348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d47b2-cdd5-4fac-8ed3-8076f2e8dfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:46.628767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d47b2-cdd5-4fac-8ed3-8076f2e8dfe6 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:46.629946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d47b2-cdd5-4fac-8ed3-8076f2e8dfe6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:46.630289 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d47b2-cdd5-4fac-8ed3-8076f2e8dfe6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:46.632147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d47b2-cdd5-4fac-8ed3-8076f2e8dfe6 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:46.656619 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 614/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Sat Jun 25 15:56:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:46.664578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ff9e3-96f0-4279-9bca-891dd47ddafc None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:46.771757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ff9e3-96f0-4279-9bca-891dd47ddafc None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:46.811758 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affdd43-7854-4762-b497-f567c3c88a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:46.812527 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affdd43-7854-4762-b497-f567c3c88a10 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:46.812846 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affdd43-7854-4762-b497-f567c3c88a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:46.813179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affdd43-7854-4762-b497-f567c3c88a10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:46.886128 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:46.893700 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affdd43-7854-4762-b497-f567c3c88a10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:46.895174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affdd43-7854-4762-b497-f567c3c88a10 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:47.003863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 627/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:56:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:56:47.010046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d47b2-cdd5-4fac-8ed3-8076f2e8dfe6 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:47.040694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:47.125527 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d47b2-cdd5-4fac-8ed3-8076f2e8dfe6 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:47.163133 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27a1c2-b302-4206-95b0-273cce6663d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:47.163954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27a1c2-b302-4206-95b0-273cce6663d4 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:47.164236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27a1c2-b302-4206-95b0-273cce6663d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:47.164540 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27a1c2-b302-4206-95b0-273cce6663d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:47.175707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27a1c2-b302-4206-95b0-273cce6663d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:47.177203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27a1c2-b302-4206-95b0-273cce6663d4 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:47.195413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-ea27a1c2-b302-4206-95b0-273cce6663d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:47.195413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:47.195413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:47.195413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:47.195413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:47.195413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:47.195413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:47.195413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:47.195413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:47.195413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:47.195413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:47.195413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:47.195413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:56:47.195413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:56:47.195413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:56:47.196626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:56:47.196626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:47.196626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:47.196626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:47.196626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:47.196626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:47.196626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:47.196626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:47.196626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:47.196626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:47.196626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:47.196626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:47.196626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:47.196626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:47.196626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:47.196626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:47.196626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:47.197706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:56:47.197706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:56:47.197706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:47.197706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:56:47.197706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 615/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:56:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:47.202922 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affdd43-7854-4762-b497-f567c3c88a10 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:47.308938 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:47.315351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affdd43-7854-4762-b497-f567c3c88a10 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:47.353589 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee530548-5d6f-42ad-8784-fb34c3554e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:47.354306 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee530548-5d6f-42ad-8784-fb34c3554e75 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:47.354632 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee530548-5d6f-42ad-8784-fb34c3554e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:47.354858 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee530548-5d6f-42ad-8784-fb34c3554e75 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:47.355624 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee530548-5d6f-42ad-8784-fb34c3554e75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:47.355929 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee530548-5d6f-42ad-8784-fb34c3554e75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:47.357368 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee530548-5d6f-42ad-8784-fb34c3554e75 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:47.369399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 628/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:47.375299 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27a1c2-b302-4206-95b0-273cce6663d4 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:47.469108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:47.492580 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27a1c2-b302-4206-95b0-273cce6663d4 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:47.530124 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f233a-485d-4c5a-949f-d78db13e47e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:47.530995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f233a-485d-4c5a-949f-d78db13e47e8 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:47.531231 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f233a-485d-4c5a-949f-d78db13e47e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:47.531497 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f233a-485d-4c5a-949f-d78db13e47e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:47.539113 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f233a-485d-4c5a-949f-d78db13e47e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:47.540928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f233a-485d-4c5a-949f-d78db13e47e8 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:47.556995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-489f233a-485d-4c5a-949f-d78db13e47e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:56:47.556995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:47.556995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:47.556995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:47.556995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:47.556995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:47.556995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:47.556995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:47.556995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:47.556995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:47.556995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:47.556995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:47.556995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:56:47.556995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:56:47.556995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:56:47.558861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:56:47.558861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:47.558861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:47.558861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:47.558861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:47.558861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:47.558861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:47.558861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:47.558861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:47.558861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:47.558861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:47.558861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:47.558861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:47.558861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:47.558861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:47.558861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:47.558861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:47.560617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:56:47.560617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:56:47.560617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:56:47.560617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:56:47.560617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:56:47.560617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:56:47.560617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:56:47.560617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:56:47.560617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 616/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:56:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:47.564369 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee530548-5d6f-42ad-8784-fb34c3554e75 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:47.681224 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee530548-5d6f-42ad-8784-fb34c3554e75 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:47.718125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf2bd69-045c-45b3-b50b-b63691b5c2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:47.718801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2bd69-045c-45b3-b50b-b63691b5c2af None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:47.719482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2bd69-045c-45b3-b50b-b63691b5c2af None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:47.719715 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2bd69-045c-45b3-b50b-b63691b5c2af None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:47.720698 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf2bd69-045c-45b3-b50b-b63691b5c2af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:47.721074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf2bd69-045c-45b3-b50b-b63691b5c2af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:47.722678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf2bd69-045c-45b3-b50b-b63691b5c2af None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:47.737849 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 629/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:56:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:56:47.744295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f233a-485d-4c5a-949f-d78db13e47e8 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:47.851708 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f233a-485d-4c5a-949f-d78db13e47e8 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:47.891526 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca79fea-bcc2-412c-8c99-07d065ad0418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:47.892334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca79fea-bcc2-412c-8c99-07d065ad0418 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:47.892715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca79fea-bcc2-412c-8c99-07d065ad0418 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:47.892987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca79fea-bcc2-412c-8c99-07d065ad0418 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:47.893821 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca79fea-bcc2-412c-8c99-07d065ad0418 None admin] RBAC: Adding query filter params (role.id=7f1996bcf7e54638b4f53b3d8ba83495, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=474c468a610140f8a5b18d5a1ed1d18e) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:47.894127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca79fea-bcc2-412c-8c99-07d065ad0418 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:47.895562 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca79fea-bcc2-412c-8c99-07d065ad0418 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:47.903019 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 617/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:56:47 2022] GET /identity/v3/role_assignments?user.id=474c468a610140f8a5b18d5a1ed1d18e&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=7f1996bcf7e54638b4f53b3d8ba83495 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:56:50.070489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe3045-8fa0-4e0b-acfb-7c692e5a1427 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:50.071163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe3045-8fa0-4e0b-acfb-7c692e5a1427 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:50.071163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe3045-8fa0-4e0b-acfb-7c692e5a1427 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:50.072599 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 630/1247] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:56:50.078469 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f97b63-e723-4f8e-a72a-b1a9d03d912f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:50.078913 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f97b63-e723-4f8e-a72a-b1a9d03d912f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:50.079025 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f97b63-e723-4f8e-a72a-b1a9d03d912f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:50.084601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:50.098382 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a6f97b63-e723-4f8e-a72a-b1a9d03d912f None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:56:50.099598 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a6f97b63-e723-4f8e-a72a-b1a9d03d912f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:56:50.187327 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 618/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:56:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:56:50.224148 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe3045-8fa0-4e0b-acfb-7c692e5a1427 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:50.248962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:50.348210 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe3045-8fa0-4e0b-acfb-7c692e5a1427 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:50.384596 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da986d-51ed-4764-b7b1-72e216968c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:50.385402 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da986d-51ed-4764-b7b1-72e216968c62 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:50.385676 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da986d-51ed-4764-b7b1-72e216968c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:50.385951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da986d-51ed-4764-b7b1-72e216968c62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:50.399052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da986d-51ed-4764-b7b1-72e216968c62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:56:50.409569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-a8da986d-51ed-4764-b7b1-72e216968c62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:56:50.409569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:50.409569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:50.409569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:50.409569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:50.409569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:50.409569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:50.409569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:50.409569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:50.409569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:50.409569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:50.409569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:50.409569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:56:50.409569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:56:50.409569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:56:50.409569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:56:50.410839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:56:50.410839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:56:50.410839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:56:50.410839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:56:50.410839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:50.410839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:50.410839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:50.410839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:50.410839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:50.410839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:50.410839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:50.410839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:50.410839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:50.410839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:50.410839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:50.410839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:50.411882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:50.411882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:50.411882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:50.411882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:50.411882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:56:50.411882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:56:50.411882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:56:50.411882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:56:50.411882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:56:50.411882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:56:50.411882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:56:50.411882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:56:50.411882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 631/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:56:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:50.419057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f97b63-e723-4f8e-a72a-b1a9d03d912f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:50.537871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f97b63-e723-4f8e-a72a-b1a9d03d912f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:50.571718 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:50.577930 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b78dcd-7ffe-4470-abc6-392404c62507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:50.578715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b78dcd-7ffe-4470-abc6-392404c62507 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:50.578993 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b78dcd-7ffe-4470-abc6-392404c62507 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:50.579323 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b78dcd-7ffe-4470-abc6-392404c62507 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:50.580200 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b78dcd-7ffe-4470-abc6-392404c62507 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:50.580555 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b78dcd-7ffe-4470-abc6-392404c62507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:50.582044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b78dcd-7ffe-4470-abc6-392404c62507 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:50.591688 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 619/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:56:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:56:50.597666 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da986d-51ed-4764-b7b1-72e216968c62 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:50.692504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:50.719492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da986d-51ed-4764-b7b1-72e216968c62 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:50.758048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a1a35-4367-4689-bbe8-1780fe861569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:50.758917 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a1a35-4367-4689-bbe8-1780fe861569 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:50.759200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a1a35-4367-4689-bbe8-1780fe861569 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:50.759472 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a1a35-4367-4689-bbe8-1780fe861569 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:50.773869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a1a35-4367-4689-bbe8-1780fe861569 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:50.775566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a1a35-4367-4689-bbe8-1780fe861569 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:50.793098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-1a6a1a35-4367-4689-bbe8-1780fe861569 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:50.793098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:50.793098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:50.793098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:50.793098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:50.793098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:50.793098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:50.793098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:50.793098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:50.793098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:50.793098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:50.793098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:50.793098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:56:50.793098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:56:50.793098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:56:50.794391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:56:50.794391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:50.794391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:50.794391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:50.794391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:50.794391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:50.794391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:50.794391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:50.794391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:50.794391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:50.794391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:50.794391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:50.794391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:50.794391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:50.794391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:50.794391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:50.794391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:50.795876 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:56:50.795876 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:56:50.795876 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:50.795876 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:56:50.795876 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 632/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:56:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:50.800450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b78dcd-7ffe-4470-abc6-392404c62507 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:50.918677 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b78dcd-7ffe-4470-abc6-392404c62507 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:50.961844 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b72a5-f872-4e43-a8e9-1724526d98f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:50.962653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b72a5-f872-4e43-a8e9-1724526d98f3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:50.962961 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b72a5-f872-4e43-a8e9-1724526d98f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:50.963227 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b72a5-f872-4e43-a8e9-1724526d98f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:50.963951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b72a5-f872-4e43-a8e9-1724526d98f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:50.964254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b72a5-f872-4e43-a8e9-1724526d98f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:50.965663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b72a5-f872-4e43-a8e9-1724526d98f3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:50.978580 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 620/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 15:56:50.984574 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a1a35-4367-4689-bbe8-1780fe861569 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:51.101141 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a1a35-4367-4689-bbe8-1780fe861569 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:51.129598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:51.138219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be79fd-6a30-488a-a615-d70d947cabfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:51.139157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be79fd-6a30-488a-a615-d70d947cabfd None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:51.139509 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be79fd-6a30-488a-a615-d70d947cabfd None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:51.139787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be79fd-6a30-488a-a615-d70d947cabfd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:51.148596 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be79fd-6a30-488a-a615-d70d947cabfd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:51.150304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be79fd-6a30-488a-a615-d70d947cabfd None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:51.165824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-01be79fd-6a30-488a-a615-d70d947cabfd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 25 15:56:51.165824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:51.165824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:51.165824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:51.165824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:51.165824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:51.165824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:51.165824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:51.165824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:51.165824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:51.165824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:51.165824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:51.165824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:56:51.165824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:56:51.165824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:56:51.165824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:56:51.167021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:51.167021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:51.167021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:56:51.167021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:56:51.167021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:56:51.167021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:56:51.167021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:51.167021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:51.167021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:51.167021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:51.167021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:51.167021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:51.167021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:51.167021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:51.167021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:51.167021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:51.167021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:51.167021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:51.168309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:51.168309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:51.168309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:56:51.168309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:56:51.168309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:56:51.168309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:56:51.168309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:56:51.168309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:56:51.168309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 25 15:56:51.168309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:56:51.168309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 633/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:56:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:51.172945 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b72a5-f872-4e43-a8e9-1724526d98f3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:51.232951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:51.283412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b72a5-f872-4e43-a8e9-1724526d98f3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:51.320622 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682247cc-be8d-4bd6-b34a-1f7e4cb87818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:51.321390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682247cc-be8d-4bd6-b34a-1f7e4cb87818 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:51.321708 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682247cc-be8d-4bd6-b34a-1f7e4cb87818 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:51.321987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682247cc-be8d-4bd6-b34a-1f7e4cb87818 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:51.322951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682247cc-be8d-4bd6-b34a-1f7e4cb87818 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:51.323247 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682247cc-be8d-4bd6-b34a-1f7e4cb87818 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:51.324927 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682247cc-be8d-4bd6-b34a-1f7e4cb87818 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:51.347193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 621/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Sat Jun 25 15:56:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:51.353610 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be79fd-6a30-488a-a615-d70d947cabfd None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:51.466407 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be79fd-6a30-488a-a615-d70d947cabfd None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:51.503491 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b152ad-68ea-482f-991b-62f2ae352112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:51.504245 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b152ad-68ea-482f-991b-62f2ae352112 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:51.504554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b152ad-68ea-482f-991b-62f2ae352112 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:51.504831 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b152ad-68ea-482f-991b-62f2ae352112 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:51.575736 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:51.589103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b152ad-68ea-482f-991b-62f2ae352112 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:51.590599 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b152ad-68ea-482f-991b-62f2ae352112 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:51.714299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 634/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:56:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:56:51.720654 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682247cc-be8d-4bd6-b34a-1f7e4cb87818 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:51.837396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682247cc-be8d-4bd6-b34a-1f7e4cb87818 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:51.852326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:51.877010 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016ee65-a363-42e7-bdff-a2f6a29bd83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:51.877820 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016ee65-a363-42e7-bdff-a2f6a29bd83d None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:51.878096 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016ee65-a363-42e7-bdff-a2f6a29bd83d None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:51.878366 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016ee65-a363-42e7-bdff-a2f6a29bd83d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:51.889276 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016ee65-a363-42e7-bdff-a2f6a29bd83d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:51.891164 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016ee65-a363-42e7-bdff-a2f6a29bd83d None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:51.912064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-7016ee65-a363-42e7-bdff-a2f6a29bd83d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:51.912064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:51.912064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:51.912064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:51.912064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:51.912064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:51.912064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:51.912064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:51.912064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:51.912064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:51.912064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:51.912064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:51.912064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:56:51.912064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:56:51.912064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:56:51.913065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:56:51.913065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:51.913065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:51.913065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:51.913065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:51.913065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:51.913065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:51.913065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:51.913065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:51.913065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:51.913065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:51.913065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:51.913065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:51.913065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:51.913065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:51.913065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:51.913065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:51.913976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:56:51.913976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:56:51.913976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:51.913976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:56:51.913976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 622/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:56:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:51.920169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b152ad-68ea-482f-991b-62f2ae352112 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:52.011499 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:52.029676 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b152ad-68ea-482f-991b-62f2ae352112 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:52.068466 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab097f6c-d6db-4ddf-8fa2-bcde007c6d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:52.069206 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab097f6c-d6db-4ddf-8fa2-bcde007c6d36 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:52.069476 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab097f6c-d6db-4ddf-8fa2-bcde007c6d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:52.069835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab097f6c-d6db-4ddf-8fa2-bcde007c6d36 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:52.070745 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab097f6c-d6db-4ddf-8fa2-bcde007c6d36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:52.071117 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab097f6c-d6db-4ddf-8fa2-bcde007c6d36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:52.072746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab097f6c-d6db-4ddf-8fa2-bcde007c6d36 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:52.085432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 635/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:52.091529 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016ee65-a363-42e7-bdff-a2f6a29bd83d None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:52.208162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016ee65-a363-42e7-bdff-a2f6a29bd83d None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:52.247922 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d6d50-5922-43a7-b45f-023846488e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:52.248748 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d6d50-5922-43a7-b45f-023846488e38 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:52.249030 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d6d50-5922-43a7-b45f-023846488e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:52.249298 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d6d50-5922-43a7-b45f-023846488e38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:52.256348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d6d50-5922-43a7-b45f-023846488e38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:52.257820 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d6d50-5922-43a7-b45f-023846488e38 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:52.271874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-3d9d6d50-5922-43a7-b45f-023846488e38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:56:52.271874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:52.271874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:52.271874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:52.271874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:52.271874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:52.271874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:52.271874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:52.271874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:52.271874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:52.271874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:52.271874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:52.271874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:56:52.271874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:56:52.271874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:56:52.272946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:56:52.272946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:52.272946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:52.272946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:52.272946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:52.272946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:52.272946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:52.272946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:52.272946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:52.272946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:52.272946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:52.272946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:52.272946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:52.272946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:52.272946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:52.272946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:52.272946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:52.273915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:56:52.273915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:56:52.273915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:56:52.273915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:56:52.273915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:56:52.273915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:56:52.273915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:56:52.273915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:56:52.273915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 623/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:56:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:52.280737 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab097f6c-d6db-4ddf-8fa2-bcde007c6d36 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:52.387729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab097f6c-d6db-4ddf-8fa2-bcde007c6d36 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:52.424609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeac001-b446-42a1-9c9b-7ead6278caa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:52.424609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeac001-b446-42a1-9c9b-7ead6278caa9 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:52.424927 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeac001-b446-42a1-9c9b-7ead6278caa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:52.425016 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeac001-b446-42a1-9c9b-7ead6278caa9 None admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:52.425741 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeac001-b446-42a1-9c9b-7ead6278caa9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:52.426045 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeac001-b446-42a1-9c9b-7ead6278caa9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:52.427476 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeac001-b446-42a1-9c9b-7ead6278caa9 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:52.440462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 636/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:56:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:52.447339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d6d50-5922-43a7-b45f-023846488e38 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:52.476260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:52.570676 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d6d50-5922-43a7-b45f-023846488e38 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:52.613645 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f8c6b-78c3-47eb-9433-c438018e800f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:52.614328 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f8c6b-78c3-47eb-9433-c438018e800f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:52.614614 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f8c6b-78c3-47eb-9433-c438018e800f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:52.614870 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f8c6b-78c3-47eb-9433-c438018e800f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:52.615728 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f8c6b-78c3-47eb-9433-c438018e800f None admin] RBAC: Adding query filter params (role.id=b1e13c449fa64b3e88b6832bb767214b, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=474c468a610140f8a5b18d5a1ed1d18e) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:52.616048 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f8c6b-78c3-47eb-9433-c438018e800f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:52.617524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f8c6b-78c3-47eb-9433-c438018e800f None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:52.624523 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 624/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:56:52 2022] GET /identity/v3/role_assignments?user.id=474c468a610140f8a5b18d5a1ed1d18e&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:54.690016 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba00c8-38eb-4f53-aef4-0c3f1039dcde None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:54.690865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba00c8-38eb-4f53-aef4-0c3f1039dcde None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:54.690865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba00c8-38eb-4f53-aef4-0c3f1039dcde None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:54.693154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 637/1261] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:56:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:56:54.699508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff898cb-1d7f-4bc7-b982-aac0b1690219 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:54.700245 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff898cb-1d7f-4bc7-b982-aac0b1690219 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:54.700386 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff898cb-1d7f-4bc7-b982-aac0b1690219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:54.722890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-4ff898cb-1d7f-4bc7-b982-aac0b1690219 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:56:54.724081 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4ff898cb-1d7f-4bc7-b982-aac0b1690219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:56:54.804747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 625/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:56:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:56:54.827756 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba00c8-38eb-4f53-aef4-0c3f1039dcde None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:54.839979 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:54.941471 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba00c8-38eb-4f53-aef4-0c3f1039dcde None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:54.976439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e4d47-ab4d-4e78-a5a1-6f7a177ca714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:54.977245 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e4d47-ab4d-4e78-a5a1-6f7a177ca714 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:54.977519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e4d47-ab4d-4e78-a5a1-6f7a177ca714 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:54.977828 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e4d47-ab4d-4e78-a5a1-6f7a177ca714 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:54.988591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e4d47-ab4d-4e78-a5a1-6f7a177ca714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:56:54.996208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-1f2e4d47-ab4d-4e78-a5a1-6f7a177ca714 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:56:54.996208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:54.996208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:54.996208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:54.996208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:54.996208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:54.996208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:54.996208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:54.996208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:54.996208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:54.996208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:54.996208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:54.996208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:56:54.996208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:56:54.996208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:56:54.996208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:56:54.997196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:56:54.997196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:56:54.997196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:56:54.997196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:56:54.997196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:54.997196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:54.997196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:54.997196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:54.997196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:54.997196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:54.997196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:54.997196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:54.997196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:54.997196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:54.997196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:54.997196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:54.998086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:54.998086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:54.998086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:54.998086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:54.998086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:56:54.998086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:56:54.998086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:56:54.998086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:56:54.998086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:56:54.998086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:56:54.998086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:56:54.998086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:56:54.998086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 638/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:56:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:55.003933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff898cb-1d7f-4bc7-b982-aac0b1690219 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:55.023280 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:55.118255 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff898cb-1d7f-4bc7-b982-aac0b1690219 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:55.153879 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069b900-f07e-4209-a11f-4844c163c5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:55.154533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069b900-f07e-4209-a11f-4844c163c5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:55.154807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069b900-f07e-4209-a11f-4844c163c5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:55.155076 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069b900-f07e-4209-a11f-4844c163c5a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:55.155822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069b900-f07e-4209-a11f-4844c163c5a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:55.156120 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069b900-f07e-4209-a11f-4844c163c5a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:55.157525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069b900-f07e-4209-a11f-4844c163c5a1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:55.165653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 626/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:56:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:55.171792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e4d47-ab4d-4e78-a5a1-6f7a177ca714 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:55.238581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:55.274170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e4d47-ab4d-4e78-a5a1-6f7a177ca714 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:55.309784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792d16d-91bc-4d9f-a79a-f10d4676abb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:55.310574 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d16d-91bc-4d9f-a79a-f10d4676abb4 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:55.310850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d16d-91bc-4d9f-a79a-f10d4676abb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:55.311119 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d16d-91bc-4d9f-a79a-f10d4676abb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:55.321978 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792d16d-91bc-4d9f-a79a-f10d4676abb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:55.323583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792d16d-91bc-4d9f-a79a-f10d4676abb4 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:55.341786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-9792d16d-91bc-4d9f-a79a-f10d4676abb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:55.341786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:55.341786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:55.341786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:55.341786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:55.341786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:55.341786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:55.341786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:55.341786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:55.341786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:55.341786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:55.341786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:55.341786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:56:55.341786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:56:55.341786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:56:55.342834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:56:55.342834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:55.342834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:55.342834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:55.342834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:55.342834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:55.342834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:55.342834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:55.342834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:55.342834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:55.342834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:55.342834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:55.342834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:55.342834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:55.342834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:55.342834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:55.342834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:55.343898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:56:55.343898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:56:55.343898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:55.343898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:56:55.343898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 639/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:56:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:55.351550 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069b900-f07e-4209-a11f-4844c163c5a1 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:55.444894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:55.464853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069b900-f07e-4209-a11f-4844c163c5a1 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:55.499876 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd2d24-847d-42dd-9ce3-909dbd583a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:55.500455 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd2d24-847d-42dd-9ce3-909dbd583a28 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:55.500758 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd2d24-847d-42dd-9ce3-909dbd583a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:55.501037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd2d24-847d-42dd-9ce3-909dbd583a28 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:55.501758 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd2d24-847d-42dd-9ce3-909dbd583a28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:55.502061 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd2d24-847d-42dd-9ce3-909dbd583a28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:55.503442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd2d24-847d-42dd-9ce3-909dbd583a28 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:55.515260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 627/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:55.521987 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792d16d-91bc-4d9f-a79a-f10d4676abb4 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:55.638607 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792d16d-91bc-4d9f-a79a-f10d4676abb4 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:55.662900 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:55.679173 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48e0ee-9201-4af3-8931-f92e6db931b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:55.679984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48e0ee-9201-4af3-8931-f92e6db931b8 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:55.680261 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48e0ee-9201-4af3-8931-f92e6db931b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:55.680571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48e0ee-9201-4af3-8931-f92e6db931b8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:55.688305 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48e0ee-9201-4af3-8931-f92e6db931b8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:55.689774 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48e0ee-9201-4af3-8931-f92e6db931b8 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:55.703834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-6a48e0ee-9201-4af3-8931-f92e6db931b8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 25 15:56:55.703834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:55.703834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:55.703834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:55.703834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:55.703834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:55.703834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:55.703834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:55.703834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:55.703834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:55.703834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:55.703834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:55.703834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:56:55.703834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:56:55.703834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:56:55.703834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:56:55.705474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:55.705474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:55.705474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:56:55.705474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:56:55.705474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:56:55.705474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:56:55.705474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:55.705474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:55.705474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:55.705474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:55.705474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:55.705474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:55.705474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:55.705474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:55.705474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:55.705474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:55.705474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:55.705474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:55.706719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:55.706719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:55.706719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:56:55.706719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:56:55.706719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:56:55.706719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:56:55.706719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:56:55.706719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:56:55.706719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 25 15:56:55.706719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:56:55.706719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 640/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:56:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:55.710903 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd2d24-847d-42dd-9ce3-909dbd583a28 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:55.811060 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd2d24-847d-42dd-9ce3-909dbd583a28 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:55.850089 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9d176-21d0-44e5-9e6c-6c61ffb572d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:55.851042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9d176-21d0-44e5-9e6c-6c61ffb572d1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:55.851434 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9d176-21d0-44e5-9e6c-6c61ffb572d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:55.851793 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9d176-21d0-44e5-9e6c-6c61ffb572d1 None admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:55.853138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9d176-21d0-44e5-9e6c-6c61ffb572d1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:55.853521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9d176-21d0-44e5-9e6c-6c61ffb572d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:55.855527 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9d176-21d0-44e5-9e6c-6c61ffb572d1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:55.864961 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:55.874632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 628/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Sat Jun 25 15:56:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:55.882885 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48e0ee-9201-4af3-8931-f92e6db931b8 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:55.988965 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48e0ee-9201-4af3-8931-f92e6db931b8 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:56.026581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff5672-ae73-4844-9d38-2233bca2a235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:56.027460 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff5672-ae73-4844-9d38-2233bca2a235 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:56.027742 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff5672-ae73-4844-9d38-2233bca2a235 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:56.028018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff5672-ae73-4844-9d38-2233bca2a235 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:56.039079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff5672-ae73-4844-9d38-2233bca2a235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:56.040570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff5672-ae73-4844-9d38-2233bca2a235 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:56.058027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-8bff5672-ae73-4844-9d38-2233bca2a235 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:56.058027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:56.058027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:56.058027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:56.058027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:56.058027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:56.058027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:56.058027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:56.058027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:56.058027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:56.058027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:56.058027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:56.058027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:56:56.058027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:56:56.058027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:56:56.059043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:56:56.059043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:56.059043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:56.059043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:56.059043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:56.059043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:56.059043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:56.059043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:56.059043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:56.059043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:56.059043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:56.059043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:56.059043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:56.059043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:56.059043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:56.059043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:56.059043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:56.060489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:56:56.060489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:56:56.060489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:56.060489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:56:56.060489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 641/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:56:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 25 15:56:56.068520 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9d176-21d0-44e5-9e6c-6c61ffb572d1 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:56.177208 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9d176-21d0-44e5-9e6c-6c61ffb572d1 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:56.211897 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cd140-917b-4459-b10a-767078ad5be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:56.212569 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cd140-917b-4459-b10a-767078ad5be2 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:56.212847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cd140-917b-4459-b10a-767078ad5be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:56.213113 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cd140-917b-4459-b10a-767078ad5be2 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:56.213829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37cd140-917b-4459-b10a-767078ad5be2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:56.214129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37cd140-917b-4459-b10a-767078ad5be2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:56.215553 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37cd140-917b-4459-b10a-767078ad5be2 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:56.228152 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 629/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:56.237307 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff5672-ae73-4844-9d38-2233bca2a235 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:56.273789 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:56.348810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff5672-ae73-4844-9d38-2233bca2a235 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:56.383749 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35673a19-8b39-4b6a-b236-ed4afbff2324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:56.384570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35673a19-8b39-4b6a-b236-ed4afbff2324 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:56.384851 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35673a19-8b39-4b6a-b236-ed4afbff2324 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:56.385125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35673a19-8b39-4b6a-b236-ed4afbff2324 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:56.392657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35673a19-8b39-4b6a-b236-ed4afbff2324 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:56.394106 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35673a19-8b39-4b6a-b236-ed4afbff2324 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:56.408355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-35673a19-8b39-4b6a-b236-ed4afbff2324 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:56:56.408355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:56.408355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:56.408355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:56.408355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:56.408355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:56.408355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:56.408355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:56.408355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:56.408355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:56.408355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:56.408355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:56.408355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:56:56.408355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:56:56.408355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:56:56.409399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:56:56.409399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:56.409399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:56.409399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:56.409399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:56.409399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:56.409399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:56.409399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:56.409399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:56.409399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:56.409399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:56.409399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:56.409399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:56.409399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:56.409399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:56.409399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:56.409399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:56.410328 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:56:56.410328 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:56:56.410328 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:56:56.410328 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:56:56.410328 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:56:56.410328 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:56:56.410328 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:56:56.410328 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:56:56.410328 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 642/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:56:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:56.415684 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cd140-917b-4459-b10a-767078ad5be2 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:56.477427 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:56.522572 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cd140-917b-4459-b10a-767078ad5be2 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:56.560691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969e2d7-7c78-4fea-a1a0-d347db69e727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:56.561349 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969e2d7-7c78-4fea-a1a0-d347db69e727 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:56.561635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969e2d7-7c78-4fea-a1a0-d347db69e727 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:56.561915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969e2d7-7c78-4fea-a1a0-d347db69e727 None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:56.562701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969e2d7-7c78-4fea-a1a0-d347db69e727 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:56.563005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969e2d7-7c78-4fea-a1a0-d347db69e727 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:56.564505 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969e2d7-7c78-4fea-a1a0-d347db69e727 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:56.577990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 630/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:56:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:56.587191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35673a19-8b39-4b6a-b236-ed4afbff2324 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:56.695352 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:56.705972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35673a19-8b39-4b6a-b236-ed4afbff2324 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:56.742871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7022d7-2b77-4dab-875a-6cbcbb3d697c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:56.743764 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7022d7-2b77-4dab-875a-6cbcbb3d697c None admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:56.744042 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7022d7-2b77-4dab-875a-6cbcbb3d697c None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:56.744318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7022d7-2b77-4dab-875a-6cbcbb3d697c None admin] PATH_INFO: `/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/474c468a610140f8a5b18d5a1ed1d18e/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:56.746560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7022d7-2b77-4dab-875a-6cbcbb3d697c None admin] RBAC: Authorizing `identity:create_grant(project_id=3600648eb94b434b8c797d2f454e46b8, user_id=474c468a610140f8a5b18d5a1ed1d18e, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:56.748645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7022d7-2b77-4dab-875a-6cbcbb3d697c None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:56.764243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 643/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Jun 25 15:56:56 2022] PUT /identity/v3/projects/3600648eb94b434b8c797d2f454e46b8/users/474c468a610140f8a5b18d5a1ed1d18e/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 15:56:58.739077 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde1cbd-7372-4622-8516-e084a6b4c881 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:58.739945 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde1cbd-7372-4622-8516-e084a6b4c881 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:58.739945 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde1cbd-7372-4622-8516-e084a6b4c881 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:58.742033 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 631/1274] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:56:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:56:58.747221 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8c23e-80ab-4b93-9703-802b29e3b7a6 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:58.747633 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8c23e-80ab-4b93-9703-802b29e3b7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:58.747870 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8c23e-80ab-4b93-9703-802b29e3b7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:58.766459 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-44d8c23e-80ab-4b93-9703-802b29e3b7a6 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:56:58.767602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-44d8c23e-80ab-4b93-9703-802b29e3b7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:56:58.849840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 644/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:56:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:56:58.875282 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde1cbd-7372-4622-8516-e084a6b4c881 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:58.995567 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde1cbd-7372-4622-8516-e084a6b4c881 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:59.013839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:59.037562 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312c662-24ae-46c3-903d-97d5d3fb0d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:59.038387 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312c662-24ae-46c3-903d-97d5d3fb0d0f None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:59.038664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312c662-24ae-46c3-903d-97d5d3fb0d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:59.038935 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312c662-24ae-46c3-903d-97d5d3fb0d0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:59.050112 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7312c662-24ae-46c3-903d-97d5d3fb0d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 25 15:56:59.057669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-7312c662-24ae-46c3-903d-97d5d3fb0d0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:56:59.057669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:59.057669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:59.057669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:59.057669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:59.057669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:59.057669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:59.057669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:59.057669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:59.057669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:59.057669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:59.057669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:59.057669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 25 15:56:59.057669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 25 15:56:59.057669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 25 15:56:59.057669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 25 15:56:59.058797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 25 15:56:59.058797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 25 15:56:59.058797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 25 15:56:59.058797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 25 15:56:59.058797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:59.058797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:59.058797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:59.058797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:59.058797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:59.058797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:59.058797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:59.058797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:59.058797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:59.058797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:59.058797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:59.058797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:59.059912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:59.059912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:59.059912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:59.059912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:59.059912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 25 15:56:59.059912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 25 15:56:59.059912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 25 15:56:59.059912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 25 15:56:59.059912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 25 15:56:59.059912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 25 15:56:59.059912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 25 15:56:59.059912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:56:59.059912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 632/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Jun 25 15:56:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:59.067345 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8c23e-80ab-4b93-9703-802b29e3b7a6 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:59.166575 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:59.187303 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8c23e-80ab-4b93-9703-802b29e3b7a6 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:59.229440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5e7cd-d172-4ee0-ba13-175c8220a053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:59.230458 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5e7cd-d172-4ee0-ba13-175c8220a053 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:59.230889 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5e7cd-d172-4ee0-ba13-175c8220a053 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:59.231301 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5e7cd-d172-4ee0-ba13-175c8220a053 None admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:59.232492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5e7cd-d172-4ee0-ba13-175c8220a053 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:59.232937 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5e7cd-d172-4ee0-ba13-175c8220a053 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:59.235176 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5e7cd-d172-4ee0-ba13-175c8220a053 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:59.248928 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 645/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Jun 25 15:56:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:59.254860 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312c662-24ae-46c3-903d-97d5d3fb0d0f None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:59.376500 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312c662-24ae-46c3-903d-97d5d3fb0d0f None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:59.413280 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b1350-af7a-4cff-937e-68f6c16ddbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:59.414060 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b1350-af7a-4cff-937e-68f6c16ddbee None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:59.414338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b1350-af7a-4cff-937e-68f6c16ddbee None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:59.414612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b1350-af7a-4cff-937e-68f6c16ddbee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:59.425650 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b1350-af7a-4cff-937e-68f6c16ddbee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:59.427104 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b1350-af7a-4cff-937e-68f6c16ddbee None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:59.444847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-b17b1350-af7a-4cff-937e-68f6c16ddbee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:59.444847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:59.444847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:59.444847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:59.444847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:59.444847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:59.444847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:59.444847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:59.444847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:59.444847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:59.444847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:59.444847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:59.444847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:56:59.444847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:56:59.444847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:56:59.445871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:56:59.445871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:59.445871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:59.445871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:59.445871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:59.445871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:59.445871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:59.445871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:59.445871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:59.445871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:59.445871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:59.445871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:59.445871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:59.445871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:59.445871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:59.445871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:59.445871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:59.447477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:56:59.447477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:56:59.447477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:56:59.447477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:56:59.447477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 633/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:56:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:59.453373 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5e7cd-d172-4ee0-ba13-175c8220a053 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:59.563587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5e7cd-d172-4ee0-ba13-175c8220a053 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:59.603010 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2fb43-5058-4d63-a600-3c3484bc0616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:59.604207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2fb43-5058-4d63-a600-3c3484bc0616 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:59.604684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2fb43-5058-4d63-a600-3c3484bc0616 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:59.605073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2fb43-5058-4d63-a600-3c3484bc0616 None admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:59.606682 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf2fb43-5058-4d63-a600-3c3484bc0616 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:59.607013 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf2fb43-5058-4d63-a600-3c3484bc0616 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:59.608989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf2fb43-5058-4d63-a600-3c3484bc0616 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:59.624614 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 646/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:59.632263 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b1350-af7a-4cff-937e-68f6c16ddbee None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:59.666960 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:59.745314 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b1350-af7a-4cff-937e-68f6c16ddbee None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:59.783298 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed002d5-f251-4cc4-9d7b-a82bdc481c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:59.784109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed002d5-f251-4cc4-9d7b-a82bdc481c10 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:59.784387 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed002d5-f251-4cc4-9d7b-a82bdc481c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:59.784694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed002d5-f251-4cc4-9d7b-a82bdc481c10 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:59.792779 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed002d5-f251-4cc4-9d7b-a82bdc481c10 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:59.794624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed002d5-f251-4cc4-9d7b-a82bdc481c10 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:59.809032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-6ed002d5-f251-4cc4-9d7b-a82bdc481c10 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 25 15:56:59.809032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:56:59.809032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:56:59.809032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:56:59.809032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:56:59.809032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:56:59.809032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:56:59.809032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:56:59.809032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:56:59.809032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:56:59.809032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:56:59.809032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:56:59.809032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 25 15:56:59.809032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 25 15:56:59.809032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 25 15:56:59.809032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 25 15:56:59.810203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:56:59.810203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:56:59.810203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 25 15:56:59.810203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:56:59.810203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 25 15:56:59.810203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 25 15:56:59.810203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:56:59.810203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:56:59.810203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:56:59.810203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:56:59.810203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:56:59.810203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:56:59.810203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:56:59.810203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:56:59.810203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:56:59.810203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:56:59.810203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:56:59.810203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:56:59.811306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:56:59.811306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:56:59.811306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 25 15:56:59.811306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 25 15:56:59.811306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 25 15:56:59.811306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 25 15:56:59.811306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 25 15:56:59.811306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 25 15:56:59.811306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 25 15:56:59.811306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:56:59.811306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 634/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:56:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:56:59.819129 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2fb43-5058-4d63-a600-3c3484bc0616 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:56:59.825064 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:56:59.933994 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2fb43-5058-4d63-a600-3c3484bc0616 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:56:59.972024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7090f-48d1-4e0c-8142-e05c06c4173f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:56:59.972718 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7090f-48d1-4e0c-8142-e05c06c4173f None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:56:59.972995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7090f-48d1-4e0c-8142-e05c06c4173f None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:56:59.973269 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7090f-48d1-4e0c-8142-e05c06c4173f None admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:56:59.974111 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7090f-48d1-4e0c-8142-e05c06c4173f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:56:59.974416 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7090f-48d1-4e0c-8142-e05c06c4173f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:56:59.975838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7090f-48d1-4e0c-8142-e05c06c4173f None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:56:59.992838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 647/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Sat Jun 25 15:56:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:56:59.999352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed002d5-f251-4cc4-9d7b-a82bdc481c10 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:57:00.103761 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:57:00.114150 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed002d5-f251-4cc4-9d7b-a82bdc481c10 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:57:00.151107 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c76dcc-4fe6-41af-a05f-4b06050bf3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:57:00.151735 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c76dcc-4fe6-41af-a05f-4b06050bf3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:00.152011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c76dcc-4fe6-41af-a05f-4b06050bf3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:00.152274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c76dcc-4fe6-41af-a05f-4b06050bf3d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:00.220947 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c76dcc-4fe6-41af-a05f-4b06050bf3d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:57:00.222400 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c76dcc-4fe6-41af-a05f-4b06050bf3d1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:57:00.311388 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:57:00.322084 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 635/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Jun 25 15:56:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 25 15:57:00.328301 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7090f-48d1-4e0c-8142-e05c06c4173f None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:57:00.397906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:57:00.438533 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7090f-48d1-4e0c-8142-e05c06c4173f None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:57:00.474646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e5f60-f0ef-4bef-b1e5-151c623d4fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:57:00.475441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e5f60-f0ef-4bef-b1e5-151c623d4fba None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:00.475750 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e5f60-f0ef-4bef-b1e5-151c623d4fba None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:00.476055 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e5f60-f0ef-4bef-b1e5-151c623d4fba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:00.487175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e5f60-f0ef-4bef-b1e5-151c623d4fba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:57:00.488689 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e5f60-f0ef-4bef-b1e5-151c623d4fba None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:57:00.506289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-a18e5f60-f0ef-4bef-b1e5-151c623d4fba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:57:00.506289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:57:00.506289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:57:00.506289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:57:00.506289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:57:00.506289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:57:00.506289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:57:00.506289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:57:00.506289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:57:00.506289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:57:00.506289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:57:00.506289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:57:00.506289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 25 15:57:00.506289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 25 15:57:00.506289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 25 15:57:00.507322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 25 15:57:00.507322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:57:00.507322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:57:00.507322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:57:00.507322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:57:00.507322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:57:00.507322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:57:00.507322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:57:00.507322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:57:00.507322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:57:00.507322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:57:00.507322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:57:00.507322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:57:00.507322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:57:00.507322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:57:00.507322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:57:00.507322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:57:00.508396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 25 15:57:00.508396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 25 15:57:00.508396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 25 15:57:00.508396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:57:00.508396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 648/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 15:57:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:57:00.513428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c76dcc-4fe6-41af-a05f-4b06050bf3d1 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:57:00.641583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c76dcc-4fe6-41af-a05f-4b06050bf3d1 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:57:00.679969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857ddea-198f-4f02-ba6e-b4f54ffb33a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:57:00.680585 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857ddea-198f-4f02-ba6e-b4f54ffb33a1 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:00.680861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857ddea-198f-4f02-ba6e-b4f54ffb33a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:00.681130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857ddea-198f-4f02-ba6e-b4f54ffb33a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:00.681860 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857ddea-198f-4f02-ba6e-b4f54ffb33a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:57:00.682154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857ddea-198f-4f02-ba6e-b4f54ffb33a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:57:00.683529 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857ddea-198f-4f02-ba6e-b4f54ffb33a1 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:57:00.694765 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 636/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Jun 25 15:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:57:00.703206 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e5f60-f0ef-4bef-b1e5-151c623d4fba None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:57:00.805076 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e5f60-f0ef-4bef-b1e5-151c623d4fba None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:57:00.820850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:57:00.852269 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b0e82-a729-4efa-8796-fd356ee623a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:57:00.853207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b0e82-a729-4efa-8796-fd356ee623a0 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:00.853488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b0e82-a729-4efa-8796-fd356ee623a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:00.853758 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b0e82-a729-4efa-8796-fd356ee623a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:00.861622 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b0e82-a729-4efa-8796-fd356ee623a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:57:00.863072 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897b0e82-a729-4efa-8796-fd356ee623a0 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:57:00.878085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application [None req-897b0e82-a729-4efa-8796-fd356ee623a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:57:00.878085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:57:00.878085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:57:00.878085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:57:00.878085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:57:00.878085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:57:00.878085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:57:00.878085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:57:00.878085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:57:00.878085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:57:00.878085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:57:00.878085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:57:00.878085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 25 15:57:00.878085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 25 15:57:00.878085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 25 15:57:00.879157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 25 15:57:00.879157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:57:00.879157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:57:00.879157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:57:00.879157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:57:00.879157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:57:00.879157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:57:00.879157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:57:00.879157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application with Lock( Jun 25 15:57:00.879157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:57:00.879157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:57:00.879157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:57:00.879157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:57:00.879157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:57:00.879157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:57:00.879157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:57:00.879157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:57:00.880123 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 25 15:57:00.880123 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 25 15:57:00.880123 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 25 15:57:00.880123 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 25 15:57:00.880123 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 25 15:57:00.880123 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 25 15:57:00.880123 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 25 15:57:00.880123 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: ERROR keystone.server.flask.application Jun 25 15:57:00.880123 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 649/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Jun 25 15:57:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:57:00.885903 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857ddea-198f-4f02-ba6e-b4f54ffb33a1 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:57:00.943515 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:57:00.997858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857ddea-198f-4f02-ba6e-b4f54ffb33a1 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:57:01.037214 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec9cf2-f167-4f3c-842c-d39525052ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:57:01.037859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec9cf2-f167-4f3c-842c-d39525052ccf None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:01.038138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec9cf2-f167-4f3c-842c-d39525052ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:01.038413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec9cf2-f167-4f3c-842c-d39525052ccf None admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:01.039206 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec9cf2-f167-4f3c-842c-d39525052ccf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:57:01.039509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec9cf2-f167-4f3c-842c-d39525052ccf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:57:01.040999 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec9cf2-f167-4f3c-842c-d39525052ccf None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:57:01.053753 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 637/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Jun 25 15:57:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:57:01.063819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b0e82-a729-4efa-8796-fd356ee623a0 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:57:01.181360 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b0e82-a729-4efa-8796-fd356ee623a0 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:57:01.220174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057522cb-82f4-4eb5-951e-30fe49f9fc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:57:01.220793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057522cb-82f4-4eb5-951e-30fe49f9fc50 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:01.221088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057522cb-82f4-4eb5-951e-30fe49f9fc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:01.221361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057522cb-82f4-4eb5-951e-30fe49f9fc50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:01.222231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057522cb-82f4-4eb5-951e-30fe49f9fc50 None admin] RBAC: Adding query filter params (role.id=b1e13c449fa64b3e88b6832bb767214b, scope.project.id=3600648eb94b434b8c797d2f454e46b8, user.id=474c468a610140f8a5b18d5a1ed1d18e) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:57:01.222552 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057522cb-82f4-4eb5-951e-30fe49f9fc50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:57:01.224023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057522cb-82f4-4eb5-951e-30fe49f9fc50 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:57:01.232846 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 650/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Jun 25 15:57:01 2022] GET /identity/v3/role_assignments?user.id=474c468a610140f8a5b18d5a1ed1d18e&scope.project.id=3600648eb94b434b8c797d2f454e46b8&role.id=b1e13c449fa64b3e88b6832bb767214b => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:57:03.308479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e79ce-d722-413b-bdd6-b9b7fca7d7f2 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:03.309383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e79ce-d722-413b-bdd6-b9b7fca7d7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:03.309383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e79ce-d722-413b-bdd6-b9b7fca7d7f2 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:03.311543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 638/1288] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:57:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:57:03.316836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe2c26-07d1-40f0-bffb-c4708c3ba9f5 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:03.317256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe2c26-07d1-40f0-bffb-c4708c3ba9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:03.317365 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe2c26-07d1-40f0-bffb-c4708c3ba9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:03.336230 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-98fe2c26-07d1-40f0-bffb-c4708c3ba9f5 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:57:03.337462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-98fe2c26-07d1-40f0-bffb-c4708c3ba9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:57:03.370655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:57:03.411153 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 651/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:57:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:57:03.437466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e79ce-d722-413b-bdd6-b9b7fca7d7f2 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:57:03.553108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e79ce-d722-413b-bdd6-b9b7fca7d7f2 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:57:03.564139 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:57:03.590561 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22b4a6-cfd6-4e19-ab4a-761cb880adc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:57:03.591623 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22b4a6-cfd6-4e19-ab4a-761cb880adc3 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:03.591928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22b4a6-cfd6-4e19-ab4a-761cb880adc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:03.592202 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22b4a6-cfd6-4e19-ab4a-761cb880adc3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:03.592890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22b4a6-cfd6-4e19-ab4a-761cb880adc3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:57:03.594281 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22b4a6-cfd6-4e19-ab4a-761cb880adc3 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:57:03.612127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-3a22b4a6-cfd6-4e19-ab4a-761cb880adc3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 25 15:57:03.612127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:57:03.612127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:57:03.612127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:57:03.612127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:57:03.612127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:57:03.612127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:57:03.612127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:57:03.612127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 25 15:57:03.612127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:57:03.612127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:57:03.612127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:57:03.612127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:57:03.612127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:57:03.612127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:57:03.612127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:57:03.612127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:57:03.613257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:57:03.613257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:57:03.613257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:57:03.613257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:57:03.613257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:57:03.613257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:57:03.613257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:57:03.613257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:57:03.613257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:57:03.613257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:57:03.613257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:57:03.613257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:57:03.613257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:57:03.613257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:57:03.613257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:57:03.613257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:57:03.614338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:57:03.614338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:57:03.614338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:57:03.614338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:57:03.614338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:57:03.614338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:57:03.614338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:57:03.614338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:57:03.614338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:57:03.614338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:57:03.614338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:57:03.614338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:57:03.614338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 25 15:57:03.614338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:57:03.614338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 639/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Jun 25 15:57:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:57:03.621112 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe2c26-07d1-40f0-bffb-c4708c3ba9f5 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:57:03.737622 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe2c26-07d1-40f0-bffb-c4708c3ba9f5 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:57:03.779655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c0287-3424-4007-adcc-c9bcf72d5ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:57:03.780319 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c0287-3424-4007-adcc-c9bcf72d5ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:03.780640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c0287-3424-4007-adcc-c9bcf72d5ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:03.780914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c0287-3424-4007-adcc-c9bcf72d5ac1 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:03.781635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c0287-3424-4007-adcc-c9bcf72d5ac1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:57:03.781939 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c0287-3424-4007-adcc-c9bcf72d5ac1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:57:03.783313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c0287-3424-4007-adcc-c9bcf72d5ac1 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:57:03.789329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 652/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Sat Jun 25 15:57:03 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:57:03.795355 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22b4a6-cfd6-4e19-ab4a-761cb880adc3 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:57:03.913936 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22b4a6-cfd6-4e19-ab4a-761cb880adc3 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:57:03.949268 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae008c-94f9-4267-8029-3cf3099ffb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:57:03.949929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae008c-94f9-4267-8029-3cf3099ffb1a None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:03.950215 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae008c-94f9-4267-8029-3cf3099ffb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:03.950596 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae008c-94f9-4267-8029-3cf3099ffb1a None admin] PATH_INFO: `/v3/services` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:03.951684 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae008c-94f9-4267-8029-3cf3099ffb1a None admin] RBAC: Adding query filter params (type=placement) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:57:03.952067 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae008c-94f9-4267-8029-3cf3099ffb1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:57:03.953567 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae008c-94f9-4267-8029-3cf3099ffb1a None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:57:03.958194 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 640/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Sat Jun 25 15:57:03 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:57:06.001884 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be080fdb-6e6c-4a25-9064-e06ec3d49979 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:06.002716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be080fdb-6e6c-4a25-9064-e06ec3d49979 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:06.002716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be080fdb-6e6c-4a25-9064-e06ec3d49979 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:06.005029 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 653/1293] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:57:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:57:06.012945 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff2249-9c32-4ab9-8231-3d68bfc9ff4b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:06.013369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff2249-9c32-4ab9-8231-3d68bfc9ff4b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:06.013457 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff2249-9c32-4ab9-8231-3d68bfc9ff4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:06.032271 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-9eff2249-9c32-4ab9-8231-3d68bfc9ff4b None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:57:06.033475 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9eff2249-9c32-4ab9-8231-3d68bfc9ff4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:57:06.063364 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:57:06.104682 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 641/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:57:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 25 15:57:06.129094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be080fdb-6e6c-4a25-9064-e06ec3d49979 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:57:06.166006 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:57:06.245083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be080fdb-6e6c-4a25-9064-e06ec3d49979 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:57:06.279377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a045f73-669b-405f-ba8a-19bca6fff7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:57:06.280034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a045f73-669b-405f-ba8a-19bca6fff7bb None admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:06.280316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a045f73-669b-405f-ba8a-19bca6fff7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:06.280618 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a045f73-669b-405f-ba8a-19bca6fff7bb None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:06.281336 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a045f73-669b-405f-ba8a-19bca6fff7bb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:57:06.282706 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a045f73-669b-405f-ba8a-19bca6fff7bb None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:57:06.299684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 654/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Jun 25 15:57:06 2022] POST /identity/v3/services => generated 246 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 15:57:08.482887 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23818674-e85b-4e09-a367-a39d35b47cc9 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:08.483786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23818674-e85b-4e09-a367-a39d35b47cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:08.483786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23818674-e85b-4e09-a367-a39d35b47cc9 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:08.485942 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 642/1296] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:57:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:57:08.492173 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20c0be-e097-4f40-9bef-b2a3bc15cdea None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:08.492916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20c0be-e097-4f40-9bef-b2a3bc15cdea None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:08.493029 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20c0be-e097-4f40-9bef-b2a3bc15cdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:08.518848 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ff20c0be-e097-4f40-9bef-b2a3bc15cdea None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:57:08.520028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ff20c0be-e097-4f40-9bef-b2a3bc15cdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:57:08.596348 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 655/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:57:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:57:08.622506 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23818674-e85b-4e09-a367-a39d35b47cc9 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:57:08.742155 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23818674-e85b-4e09-a367-a39d35b47cc9 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:57:08.788253 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2ec79-9a00-4429-aa6d-8cc854c45ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:57:08.789443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2ec79-9a00-4429-aa6d-8cc854c45ada None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:08.789715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2ec79-9a00-4429-aa6d-8cc854c45ada None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:08.789981 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2ec79-9a00-4429-aa6d-8cc854c45ada None admin] PATH_INFO: `/v3/services/placement` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:08.790635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2ec79-9a00-4429-aa6d-8cc854c45ada None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:57:08.792022 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2ec79-9a00-4429-aa6d-8cc854c45ada None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:57:08.804611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-40a2ec79-9a00-4429-aa6d-8cc854c45ada None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 25 15:57:08.804611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:57:08.804611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:57:08.804611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:57:08.804611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:57:08.804611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:57:08.804611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:57:08.804611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:57:08.804611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 25 15:57:08.804611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:57:08.804611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:57:08.804611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:57:08.804611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:57:08.804611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:57:08.804611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:57:08.804611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:57:08.804611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:57:08.805755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:57:08.805755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:57:08.805755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:57:08.805755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:57:08.805755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:57:08.805755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:57:08.805755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:57:08.805755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:57:08.805755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:57:08.805755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:57:08.805755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:57:08.805755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:57:08.805755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:57:08.805755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:57:08.805755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:57:08.805755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:57:08.806786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:57:08.806786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:57:08.806786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:57:08.806786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:57:08.806786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:57:08.806786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:57:08.806786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:57:08.806786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:57:08.806786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:57:08.806786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:57:08.806786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:57:08.806786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:57:08.806786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 25 15:57:08.806786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:57:08.806786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 643/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Jun 25 15:57:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:57:08.813204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20c0be-e097-4f40-9bef-b2a3bc15cdea None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:57:08.832162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:57:08.919102 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20c0be-e097-4f40-9bef-b2a3bc15cdea None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:57:08.954480 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc18690-79ce-4d9f-a032-ceaeb577adf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:57:08.955149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc18690-79ce-4d9f-a032-ceaeb577adf1 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:08.955424 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc18690-79ce-4d9f-a032-ceaeb577adf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:08.955696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc18690-79ce-4d9f-a032-ceaeb577adf1 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:08.956455 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc18690-79ce-4d9f-a032-ceaeb577adf1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:57:08.956747 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc18690-79ce-4d9f-a032-ceaeb577adf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:57:08.958136 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc18690-79ce-4d9f-a032-ceaeb577adf1 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:57:08.963297 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 656/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Sat Jun 25 15:57:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:57:08.970403 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2ec79-9a00-4429-aa6d-8cc854c45ada None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:57:09.014836 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:57:09.084543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2ec79-9a00-4429-aa6d-8cc854c45ada None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:57:09.120138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbd371-5c7b-4876-9d85-1201931b1181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:57:09.120812 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbd371-5c7b-4876-9d85-1201931b1181 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:09.121096 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbd371-5c7b-4876-9d85-1201931b1181 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:09.121364 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbd371-5c7b-4876-9d85-1201931b1181 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:09.122156 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbd371-5c7b-4876-9d85-1201931b1181 None admin] RBAC: Adding query filter params (interface=public, service_id=f81b124bd6714607896b0a3677eebb8a, region_id=RegionOne) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:57:09.122472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbd371-5c7b-4876-9d85-1201931b1181 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:57:09.123887 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbd371-5c7b-4876-9d85-1201931b1181 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:57:09.129085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 644/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Jun 25 15:57:08 2022] GET /identity/v3/endpoints?service_id=f81b124bd6714607896b0a3677eebb8a&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:57:09.137637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc18690-79ce-4d9f-a032-ceaeb577adf1 None admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:57:09.233757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:57:09.263722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc18690-79ce-4d9f-a032-ceaeb577adf1 None admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:57:09.298600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f27fca-d0d3-47b6-b05f-4716eda9f108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:57:09.299181 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f27fca-d0d3-47b6-b05f-4716eda9f108 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:09.299453 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f27fca-d0d3-47b6-b05f-4716eda9f108 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:09.299723 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f27fca-d0d3-47b6-b05f-4716eda9f108 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:09.300438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f27fca-d0d3-47b6-b05f-4716eda9f108 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:57:09.301781 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f27fca-d0d3-47b6-b05f-4716eda9f108 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:57:09.308609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 657/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Jun 25 15:57:09 2022] GET /identity/v3/services => generated 2191 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 15:57:11.327023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82703bd-6adf-4649-8971-344f0ed859d7 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:11.327949 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82703bd-6adf-4649-8971-344f0ed859d7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:11.327949 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82703bd-6adf-4649-8971-344f0ed859d7 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:11.330242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 645/1302] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:57:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:57:11.335594 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b49277-6134-477f-85f4-a50e006232c9 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:11.336024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b49277-6134-477f-85f4-a50e006232c9 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:11.336133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b49277-6134-477f-85f4-a50e006232c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:11.353542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a2b49277-6134-477f-85f4-a50e006232c9 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:57:11.354706 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a2b49277-6134-477f-85f4-a50e006232c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:57:11.424163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 658/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:57:11 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 25 15:57:11.448143 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82703bd-6adf-4649-8971-344f0ed859d7 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:57:11.549962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:57:11.562015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82703bd-6adf-4649-8971-344f0ed859d7 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:57:11.603978 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01af9e1-a043-417e-852c-8c7ea5069127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:57:11.604807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01af9e1-a043-417e-852c-8c7ea5069127 None admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:11.605075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01af9e1-a043-417e-852c-8c7ea5069127 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:11.605421 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01af9e1-a043-417e-852c-8c7ea5069127 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:11.606113 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01af9e1-a043-417e-852c-8c7ea5069127 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:57:11.607705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01af9e1-a043-417e-852c-8c7ea5069127 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:57:11.620803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application [None req-f01af9e1-a043-417e-852c-8c7ea5069127 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 25 15:57:11.620803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:57:11.620803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 25 15:57:11.620803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 25 15:57:11.620803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 25 15:57:11.620803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 25 15:57:11.620803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 25 15:57:11.620803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:57:11.620803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 25 15:57:11.620803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:57:11.620803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 25 15:57:11.620803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:57:11.620803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 25 15:57:11.620803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 25 15:57:11.620803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 25 15:57:11.620803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 25 15:57:11.620803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 25 15:57:11.622109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 25 15:57:11.622109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 25 15:57:11.622109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 25 15:57:11.622109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 25 15:57:11.622109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 25 15:57:11.622109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 25 15:57:11.622109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 25 15:57:11.622109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 25 15:57:11.622109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 25 15:57:11.622109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 25 15:57:11.622109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 25 15:57:11.622109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 25 15:57:11.622109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 25 15:57:11.622109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 25 15:57:11.622109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 25 15:57:11.622109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.get_or_create( Jun 25 15:57:11.623510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 25 15:57:11.623510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application with Lock( Jun 25 15:57:11.623510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 25 15:57:11.623510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self._enter() Jun 25 15:57:11.623510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 25 15:57:11.623510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 25 15:57:11.623510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 25 15:57:11.623510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application return self.creator() Jun 25 15:57:11.623510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 25 15:57:11.623510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application created_value = creator( Jun 25 15:57:11.623510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 25 15:57:11.623510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 25 15:57:11.623510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 25 15:57:11.623510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: ERROR keystone.server.flask.application Jun 25 15:57:11.623510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 646/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Jun 25 15:57:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 25 15:57:11.630219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b49277-6134-477f-85f4-a50e006232c9 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:57:11.714918 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:57:11.746867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b49277-6134-477f-85f4-a50e006232c9 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:57:11.782568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e27f33-7125-45cc-8711-69a611c67ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:57:11.783217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e27f33-7125-45cc-8711-69a611c67ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:11.783495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e27f33-7125-45cc-8711-69a611c67ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:11.783766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e27f33-7125-45cc-8711-69a611c67ca9 None admin] PATH_INFO: `/v3/services` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:11.784452 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e27f33-7125-45cc-8711-69a611c67ca9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 15:57:11.784792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e27f33-7125-45cc-8711-69a611c67ca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:57:11.786159 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e27f33-7125-45cc-8711-69a611c67ca9 None admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:57:11.791457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 659/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Sat Jun 25 15:57:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 15:57:11.798085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01af9e1-a043-417e-852c-8c7ea5069127 None admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:57:11.919310 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01af9e1-a043-417e-852c-8c7ea5069127 None admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:57:11.956122 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549f403-97fe-49bb-b5cd-63430cee8c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:57:11.956890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549f403-97fe-49bb-b5cd-63430cee8c29 None admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:57:11.957183 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549f403-97fe-49bb-b5cd-63430cee8c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:57:11.957440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549f403-97fe-49bb-b5cd-63430cee8c29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:57:11.958186 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549f403-97fe-49bb-b5cd-63430cee8c29 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:57:11.959572 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549f403-97fe-49bb-b5cd-63430cee8c29 None admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:57:11.982348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 647/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Jun 25 15:57:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 15:57:21.788236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=30 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:57:21.978122 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:15.252588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8613c5-c23d-4309-b852-9052ded6a2f0 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:15.253428 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8613c5-c23d-4309-b852-9052ded6a2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:15.253428 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8613c5-c23d-4309-b852-9052ded6a2f0 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:15.259987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2c509-01fb-4261-8fec-53eb384e282a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:15.260699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2c509-01fb-4261-8fec-53eb384e282a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:15.260699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2c509-01fb-4261-8fec-53eb384e282a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:15.263760 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 660/1307] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:58:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:58:15.340278 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2fb2c509-01fb-4261-8fec-53eb384e282a None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:58:15.341562 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2fb2c509-01fb-4261-8fec-53eb384e282a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:15.465498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 648/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:58:15.524570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5540f2-f856-4815-a7db-9e26505f4a62 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:15.525046 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5540f2-f856-4815-a7db-9e26505f4a62 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:15.525429 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5540f2-f856-4815-a7db-9e26505f4a62 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:15.527495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 661/1309] 10.222.0.13 () {54 vars in 964 bytes} [Sat Jun 25 15:58:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:58:15.540807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d670e-c677-4892-8515-465def69449a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:15.541250 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d670e-c677-4892-8515-465def69449a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:15.541665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d670e-c677-4892-8515-465def69449a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:15.695122 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a17d670e-c677-4892-8515-465def69449a None None] MFA Rules not processed for user `aaffb9f0bf714822829455c0487376ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:58:15.696260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a17d670e-c677-4892-8515-465def69449a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:15.728655 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:15.862716 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 649/1310] 10.222.0.13 () {58 vars in 1061 bytes} [Sat Jun 25 15:58:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 25 15:58:15.865239 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5540f2-f856-4815-a7db-9e26505f4a62 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:58:15.867090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-fd5540f2-f856-4815-a7db-9e26505f4a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:16.004042 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5540f2-f856-4815-a7db-9e26505f4a62 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:58:16.086690 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df9039-c72c-40f1-a03f-2c9cb3ae2c65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:58:16.087513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df9039-c72c-40f1-a03f-2c9cb3ae2c65 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:16.087803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df9039-c72c-40f1-a03f-2c9cb3ae2c65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:16.088082 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df9039-c72c-40f1-a03f-2c9cb3ae2c65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:16.089767 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f7df9039-c72c-40f1-a03f-2c9cb3ae2c65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:16.151122 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:16.214181 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df9039-c72c-40f1-a03f-2c9cb3ae2c65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:58:16.215663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df9039-c72c-40f1-a03f-2c9cb3ae2c65 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:58:16.236521 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 662/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:58:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:58:18.597431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0dd2d-1f44-4c87-8e55-a130575cbd93 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:18.597998 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0dd2d-1f44-4c87-8e55-a130575cbd93 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:18.597998 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0dd2d-1f44-4c87-8e55-a130575cbd93 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:18.599365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 650/1312] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:58:18.604849 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2632a09-579e-4785-8dc4-630d7e446d76 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:18.605297 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2632a09-579e-4785-8dc4-630d7e446d76 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:18.605389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2632a09-579e-4785-8dc4-630d7e446d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:18.690268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e2632a09-579e-4785-8dc4-630d7e446d76 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:58:18.691485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e2632a09-579e-4785-8dc4-630d7e446d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:18.791056 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 663/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:58:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:58:18.830085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0dd2d-1f44-4c87-8e55-a130575cbd93 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:58:18.831639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-22b0dd2d-1f44-4c87-8e55-a130575cbd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:18.940966 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0dd2d-1f44-4c87-8e55-a130575cbd93 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:58:19.025682 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:19.029345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b8f75-9edf-4061-8208-eff6e45d7ead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:58:19.030020 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b8f75-9edf-4061-8208-eff6e45d7ead service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:19.030311 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b8f75-9edf-4061-8208-eff6e45d7ead service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:19.030587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b8f75-9edf-4061-8208-eff6e45d7ead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:19.033461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d31b8f75-9edf-4061-8208-eff6e45d7ead service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:19.154752 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b8f75-9edf-4061-8208-eff6e45d7ead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:58:19.156215 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b8f75-9edf-4061-8208-eff6e45d7ead service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:58:19.174229 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 651/1314] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:58:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:58:21.505820 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455859a-b3d3-4beb-9fd3-215ea7401acc None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:21.506416 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455859a-b3d3-4beb-9fd3-215ea7401acc None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:21.506416 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455859a-b3d3-4beb-9fd3-215ea7401acc None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:21.507776 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 664/1315] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:58:21.514160 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d0326-5141-49fd-8d1e-f48fea797245 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:21.514881 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d0326-5141-49fd-8d1e-f48fea797245 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:21.515026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d0326-5141-49fd-8d1e-f48fea797245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:21.592939 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b87d0326-5141-49fd-8d1e-f48fea797245 None None] MFA Rules not processed for user `0233db48571f40ee92089c7c9fc98814`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:58:21.594158 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b87d0326-5141-49fd-8d1e-f48fea797245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:21.649135 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:21.696185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 652/1316] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:58:21 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:58:21.737850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455859a-b3d3-4beb-9fd3-215ea7401acc None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:58:21.739410 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8455859a-b3d3-4beb-9fd3-215ea7401acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:21.750372 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:21.852657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455859a-b3d3-4beb-9fd3-215ea7401acc None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:58:21.931448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ef80c-4240-4204-bcc5-76fa7722c362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:58:21.932088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ef80c-4240-4204-bcc5-76fa7722c362 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:21.932372 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ef80c-4240-4204-bcc5-76fa7722c362 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:21.932678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ef80c-4240-4204-bcc5-76fa7722c362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:21.934319 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-607ef80c-4240-4204-bcc5-76fa7722c362 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:21.998881 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:22.047641 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ef80c-4240-4204-bcc5-76fa7722c362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:58:22.049466 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ef80c-4240-4204-bcc5-76fa7722c362 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:58:22.068171 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 665/1317] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:58:21 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:58:24.646177 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5bdeb-a6b5-4bfa-a27b-d4334f491cad None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:24.646766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5bdeb-a6b5-4bfa-a27b-d4334f491cad None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:24.646766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5bdeb-a6b5-4bfa-a27b-d4334f491cad None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:24.648119 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 653/1318] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:58:24.653467 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260107c-d433-427d-98ec-48a9936bf54e None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:24.653905 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260107c-d433-427d-98ec-48a9936bf54e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:24.654007 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260107c-d433-427d-98ec-48a9936bf54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:24.720350 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-0260107c-d433-427d-98ec-48a9936bf54e None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:58:24.721564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0260107c-d433-427d-98ec-48a9936bf54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:24.822050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 666/1319] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:58:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:58:24.861838 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5bdeb-a6b5-4bfa-a27b-d4334f491cad None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:58:24.864010 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3bb5bdeb-a6b5-4bfa-a27b-d4334f491cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:24.990621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5bdeb-a6b5-4bfa-a27b-d4334f491cad None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:58:25.072235 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6ba76-5e21-41f7-87ce-480d73b8d801 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:58:25.072903 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6ba76-5e21-41f7-87ce-480d73b8d801 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:25.073147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6ba76-5e21-41f7-87ce-480d73b8d801 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:25.073445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6ba76-5e21-41f7-87ce-480d73b8d801 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:25.075075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4eb6ba76-5e21-41f7-87ce-480d73b8d801 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:25.083024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:25.197703 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6ba76-5e21-41f7-87ce-480d73b8d801 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:58:25.199211 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6ba76-5e21-41f7-87ce-480d73b8d801 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:58:25.217568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 654/1320] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:58:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:58:27.886247 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f71df3-4732-40cb-8368-18480cb8cd1b None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:27.886713 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f71df3-4732-40cb-8368-18480cb8cd1b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:27.886831 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f71df3-4732-40cb-8368-18480cb8cd1b None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:27.888174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 667/1321] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:58:27.893854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3826c1-f347-4ad2-8821-132cf2a530f4 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:27.894293 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3826c1-f347-4ad2-8821-132cf2a530f4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:27.894380 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3826c1-f347-4ad2-8821-132cf2a530f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:27.961926 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-cc3826c1-f347-4ad2-8821-132cf2a530f4 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:58:27.963139 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-cc3826c1-f347-4ad2-8821-132cf2a530f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:28.023290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:28.068116 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 655/1322] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:58:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:58:28.120522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfb176-e0f9-4b8a-8f68-03ba9ef6368d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:28.120861 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfb176-e0f9-4b8a-8f68-03ba9ef6368d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:28.121236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfb176-e0f9-4b8a-8f68-03ba9ef6368d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:28.122736 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 668/1323] 10.222.0.13 () {54 vars in 964 bytes} [Sat Jun 25 15:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:58:28.129703 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3514c-4dfc-43f3-8320-0d4516ed2f22 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:28.129978 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3514c-4dfc-43f3-8320-0d4516ed2f22 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:28.130261 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3514c-4dfc-43f3-8320-0d4516ed2f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:28.216122 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-97e3514c-4dfc-43f3-8320-0d4516ed2f22 None None] MFA Rules not processed for user `aaffb9f0bf714822829455c0487376ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:58:28.217458 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-97e3514c-4dfc-43f3-8320-0d4516ed2f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:28.304893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 656/1324] 10.222.0.13 () {58 vars in 1061 bytes} [Sat Jun 25 15:58:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:58:28.314430 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfb176-e0f9-4b8a-8f68-03ba9ef6368d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:58:28.316073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-fccfb176-e0f9-4b8a-8f68-03ba9ef6368d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:28.336788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:28.436604 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfb176-e0f9-4b8a-8f68-03ba9ef6368d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:58:28.513378 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdb785-6b66-4d66-bfbe-1cf8876645d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:58:28.513942 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdb785-6b66-4d66-bfbe-1cf8876645d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:28.514220 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdb785-6b66-4d66-bfbe-1cf8876645d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:28.514492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdb785-6b66-4d66-bfbe-1cf8876645d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:28.516706 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5efdb785-6b66-4d66-bfbe-1cf8876645d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:28.587635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:28.628133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdb785-6b66-4d66-bfbe-1cf8876645d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:58:28.629599 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdb785-6b66-4d66-bfbe-1cf8876645d8 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:58:28.648240 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 669/1325] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:58:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:58:30.919080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe6239-2db6-4395-b409-cdf13ec5ac35 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:30.919695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe6239-2db6-4395-b409-cdf13ec5ac35 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:30.919695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe6239-2db6-4395-b409-cdf13ec5ac35 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:30.921056 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 657/1326] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:58:30.926570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f3473-d65f-4f86-9f4d-e7b35eaafb47 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:30.927007 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f3473-d65f-4f86-9f4d-e7b35eaafb47 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:30.927124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f3473-d65f-4f86-9f4d-e7b35eaafb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:30.993727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-d61f3473-d65f-4f86-9f4d-e7b35eaafb47 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:58:30.994931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d61f3473-d65f-4f86-9f4d-e7b35eaafb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:31.098158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 670/1327] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:58:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:58:31.133842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe6239-2db6-4395-b409-cdf13ec5ac35 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:58:31.135417 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-74fe6239-2db6-4395-b409-cdf13ec5ac35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:31.200947 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:31.245203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe6239-2db6-4395-b409-cdf13ec5ac35 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:58:31.321366 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5834f65-7717-49a7-b7ba-3ff164e0f2af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:58:31.322021 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5834f65-7717-49a7-b7ba-3ff164e0f2af service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:31.322299 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5834f65-7717-49a7-b7ba-3ff164e0f2af service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:31.322577 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5834f65-7717-49a7-b7ba-3ff164e0f2af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:31.324220 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d5834f65-7717-49a7-b7ba-3ff164e0f2af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:31.436239 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:31.438836 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5834f65-7717-49a7-b7ba-3ff164e0f2af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:58:31.440838 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5834f65-7717-49a7-b7ba-3ff164e0f2af service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:58:31.459371 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 658/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:58:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:58:33.630878 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3bd32-cd3c-435f-988b-e52af656dd0a None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:33.631357 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3bd32-cd3c-435f-988b-e52af656dd0a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:33.631357 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3bd32-cd3c-435f-988b-e52af656dd0a None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:33.632699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 671/1329] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:58:33.638251 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c620b8-12f2-4560-8b67-307d25daef42 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:33.638703 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c620b8-12f2-4560-8b67-307d25daef42 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:33.638812 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c620b8-12f2-4560-8b67-307d25daef42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:33.705339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-52c620b8-12f2-4560-8b67-307d25daef42 None None] MFA Rules not processed for user `0233db48571f40ee92089c7c9fc98814`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:58:33.706595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-52c620b8-12f2-4560-8b67-307d25daef42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:33.826730 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 659/1330] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:58:33 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:58:33.865425 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3bd32-cd3c-435f-988b-e52af656dd0a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:58:33.866989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-92c3bd32-cd3c-435f-988b-e52af656dd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:33.902829 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:33.980995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3bd32-cd3c-435f-988b-e52af656dd0a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:58:34.060709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f704a13-07d0-4444-9894-dc08d8de3387 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:58:34.061261 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f704a13-07d0-4444-9894-dc08d8de3387 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:34.061539 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f704a13-07d0-4444-9894-dc08d8de3387 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:34.061817 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f704a13-07d0-4444-9894-dc08d8de3387 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:34.063524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6f704a13-07d0-4444-9894-dc08d8de3387 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:34.168087 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:34.202025 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f704a13-07d0-4444-9894-dc08d8de3387 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:58:34.203459 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f704a13-07d0-4444-9894-dc08d8de3387 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:58:34.222678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 672/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:58:33 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:58:36.490002 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a9c6c-5840-4bb4-beac-e5e2161da83d None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:36.490917 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a9c6c-5840-4bb4-beac-e5e2161da83d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:36.490917 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a9c6c-5840-4bb4-beac-e5e2161da83d None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:36.493156 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 660/1332] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:58:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:58:36.498627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa47d56-f868-4926-aed5-36aec69e90b2 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:36.499051 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa47d56-f868-4926-aed5-36aec69e90b2 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:36.499168 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa47d56-f868-4926-aed5-36aec69e90b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:36.579742 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1fa47d56-f868-4926-aed5-36aec69e90b2 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:58:36.581001 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1fa47d56-f868-4926-aed5-36aec69e90b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:36.683412 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 673/1333] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:58:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:58:36.718800 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a9c6c-5840-4bb4-beac-e5e2161da83d None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:58:36.721244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c80a9c6c-5840-4bb4-beac-e5e2161da83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:36.803987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:36.840103 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a9c6c-5840-4bb4-beac-e5e2161da83d None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:58:36.914467 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ea7fd-6410-4eb7-8985-832f8e52bbe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:58:36.915097 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ea7fd-6410-4eb7-8985-832f8e52bbe0 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:36.915394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ea7fd-6410-4eb7-8985-832f8e52bbe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:36.915668 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ea7fd-6410-4eb7-8985-832f8e52bbe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:36.917351 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-579ea7fd-6410-4eb7-8985-832f8e52bbe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:37.034721 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ea7fd-6410-4eb7-8985-832f8e52bbe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:58:37.036189 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ea7fd-6410-4eb7-8985-832f8e52bbe0 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:58:37.047487 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:37.057075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 661/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:58:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:58:39.129755 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dcfbe-6e87-402b-9069-358d7c26143f None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:39.130602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dcfbe-6e87-402b-9069-358d7c26143f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:39.130602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dcfbe-6e87-402b-9069-358d7c26143f None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:39.132454 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 674/1335] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:58:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:58:39.137932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9f355-ce2c-40e6-8088-d67ee92f842f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:39.138352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9f355-ce2c-40e6-8088-d67ee92f842f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:39.138462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9f355-ce2c-40e6-8088-d67ee92f842f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:39.209849 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-22f9f355-ce2c-40e6-8088-d67ee92f842f None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:58:39.211079 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-22f9f355-ce2c-40e6-8088-d67ee92f842f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:39.316273 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 662/1336] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:58:39.360498 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75dcfbe-6e87-402b-9069-358d7c26143f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:58:39.362902 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e75dcfbe-6e87-402b-9069-358d7c26143f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:39.410510 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:39.480105 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75dcfbe-6e87-402b-9069-358d7c26143f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:58:39.567172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec65d7-8327-4e1c-b9a3-c7f04a4517e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:58:39.567814 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec65d7-8327-4e1c-b9a3-c7f04a4517e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:39.568077 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec65d7-8327-4e1c-b9a3-c7f04a4517e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:39.568361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec65d7-8327-4e1c-b9a3-c7f04a4517e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:39.570018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5cec65d7-8327-4e1c-b9a3-c7f04a4517e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:39.670229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:39.695987 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec65d7-8327-4e1c-b9a3-c7f04a4517e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:58:39.697477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec65d7-8327-4e1c-b9a3-c7f04a4517e2 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:58:39.717256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 675/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:58:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:58:42.097382 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3b0f6-604f-4d30-85f1-2225166d898f None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:42.097944 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3b0f6-604f-4d30-85f1-2225166d898f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:42.097944 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3b0f6-604f-4d30-85f1-2225166d898f None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:42.099283 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 663/1338] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:58:42.105822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e565c40-0df9-48b2-a622-8aeee04b4c2d None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:42.106544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e565c40-0df9-48b2-a622-8aeee04b4c2d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:42.106706 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e565c40-0df9-48b2-a622-8aeee04b4c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:42.174781 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5e565c40-0df9-48b2-a622-8aeee04b4c2d None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:58:42.175932 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5e565c40-0df9-48b2-a622-8aeee04b4c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:42.276288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 676/1339] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:58:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:58:42.320368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3b0f6-604f-4d30-85f1-2225166d898f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:58:42.321914 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f7d3b0f6-604f-4d30-85f1-2225166d898f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:42.398732 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:42.428200 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3b0f6-604f-4d30-85f1-2225166d898f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:58:42.502255 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2a077-d6a3-413b-9a6d-8eeca45e0b17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:58:42.502898 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2a077-d6a3-413b-9a6d-8eeca45e0b17 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:42.503169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2a077-d6a3-413b-9a6d-8eeca45e0b17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:42.503437 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2a077-d6a3-413b-9a6d-8eeca45e0b17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:42.505105 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d8a2a077-d6a3-413b-9a6d-8eeca45e0b17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:42.636296 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a2a077-d6a3-413b-9a6d-8eeca45e0b17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:58:42.637789 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a2a077-d6a3-413b-9a6d-8eeca45e0b17 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:58:42.657340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 664/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:58:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:58:46.968977 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45299de7-2445-4062-9b5d-c89998599b50 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:46.970234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45299de7-2445-4062-9b5d-c89998599b50 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:46.970234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45299de7-2445-4062-9b5d-c89998599b50 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:46.971099 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 677/1341] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:58:46.976314 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bb575-383b-499d-b106-ed412f9c6f45 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:46.976775 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bb575-383b-499d-b106-ed412f9c6f45 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:46.976914 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bb575-383b-499d-b106-ed412f9c6f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:46.984090 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:47.056851 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-692bb575-383b-499d-b106-ed412f9c6f45 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:58:47.058074 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-692bb575-383b-499d-b106-ed412f9c6f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:47.167863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 665/1342] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:58:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:58:47.206069 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45299de7-2445-4062-9b5d-c89998599b50 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:58:47.208309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-45299de7-2445-4062-9b5d-c89998599b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:47.261677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:47.310059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45299de7-2445-4062-9b5d-c89998599b50 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:58:47.314281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e403670-5b0f-4b25-9e8d-420321b07b57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:58:47.314937 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e403670-5b0f-4b25-9e8d-420321b07b57 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:47.315217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e403670-5b0f-4b25-9e8d-420321b07b57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:47.315493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e403670-5b0f-4b25-9e8d-420321b07b57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:47.325270 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e403670-5b0f-4b25-9e8d-420321b07b57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:58:47.327136 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e403670-5b0f-4b25-9e8d-420321b07b57 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:58:47.338211 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 678/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:58:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:58:50.008948 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a25e3-8925-40ad-b6b6-ca3336500d89 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:50.009488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a25e3-8925-40ad-b6b6-ca3336500d89 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:50.009488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a25e3-8925-40ad-b6b6-ca3336500d89 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:50.011031 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 666/1344] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:58:50.016963 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e11895-b54e-4d88-beae-d78c01a0c8bc None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:50.017414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e11895-b54e-4d88-beae-d78c01a0c8bc None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:50.017522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e11895-b54e-4d88-beae-d78c01a0c8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:50.047014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-55e11895-b54e-4d88-beae-d78c01a0c8bc None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:58:50.048264 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-55e11895-b54e-4d88-beae-d78c01a0c8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:50.135649 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 679/1345] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:58:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:58:50.173806 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a25e3-8925-40ad-b6b6-ca3336500d89 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:58:50.175579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-600a25e3-8925-40ad-b6b6-ca3336500d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:50.189665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a25e3-8925-40ad-b6b6-ca3336500d89 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:58:50.193570 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca11f5e-105d-4ebf-a8ba-d62303d961c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:58:50.194237 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca11f5e-105d-4ebf-a8ba-d62303d961c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:50.194516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca11f5e-105d-4ebf-a8ba-d62303d961c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:50.194787 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca11f5e-105d-4ebf-a8ba-d62303d961c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:50.205700 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca11f5e-105d-4ebf-a8ba-d62303d961c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:58:50.207192 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca11f5e-105d-4ebf-a8ba-d62303d961c3 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:58:50.218165 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 667/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:58:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:58:54.506631 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfdee6-7841-414f-8fad-8b37d3095c7d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:54.507118 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfdee6-7841-414f-8fad-8b37d3095c7d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:54.507206 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdfdee6-7841-414f-8fad-8b37d3095c7d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:54.508786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 680/1347] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:58:54.513832 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f8439-f030-45b3-b2ce-7809a7130f05 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:54.514281 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f8439-f030-45b3-b2ce-7809a7130f05 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:54.514369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f8439-f030-45b3-b2ce-7809a7130f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:54.532821 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-da1f8439-f030-45b3-b2ce-7809a7130f05 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:58:54.534012 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-da1f8439-f030-45b3-b2ce-7809a7130f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:54.542499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 668/1348] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:58:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:58:54.581490 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdfdee6-7841-414f-8fad-8b37d3095c7d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:58:54.592731 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdfdee6-7841-414f-8fad-8b37d3095c7d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:58:54.597043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6935f6be-4c7c-4371-a11f-362505fcbe2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:58:54.597696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935f6be-4c7c-4371-a11f-362505fcbe2e service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:54.597979 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935f6be-4c7c-4371-a11f-362505fcbe2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:54.598258 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935f6be-4c7c-4371-a11f-362505fcbe2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:54.607509 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935f6be-4c7c-4371-a11f-362505fcbe2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:58:54.609109 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6935f6be-4c7c-4371-a11f-362505fcbe2e service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:58:54.621413 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 681/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:58:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:58:59.580612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3f141-74e6-4b5f-bf3f-e3efe2bac19d None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:59.581285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3f141-74e6-4b5f-bf3f-e3efe2bac19d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:59.581285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3f141-74e6-4b5f-bf3f-e3efe2bac19d None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:59.582515 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 669/1350] 10.222.0.13 () {54 vars in 921 bytes} [Sat Jun 25 15:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:58:59.587770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b689b3a-e851-4f21-8485-35aede8b0f80 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:58:59.588553 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b689b3a-e851-4f21-8485-35aede8b0f80 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:58:59.589089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b689b3a-e851-4f21-8485-35aede8b0f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:58:59.669030 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7b689b3a-e851-4f21-8485-35aede8b0f80 None None] MFA Rules not processed for user `474c468a610140f8a5b18d5a1ed1d18e`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:58:59.670551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7b689b3a-e851-4f21-8485-35aede8b0f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:58:59.698634 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:58:59.759313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 682/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Jun 25 15:58:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:04.198562 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662f7bf-b6cb-4de1-95b5-a60f21fd67b6 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:04.199157 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662f7bf-b6cb-4de1-95b5-a60f21fd67b6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:04.199157 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7662f7bf-b6cb-4de1-95b5-a60f21fd67b6 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:04.200690 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 670/1352] 10.222.0.13 () {54 vars in 921 bytes} [Sat Jun 25 15:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:04.206077 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759ad98-778c-4246-8ca3-f57d163fa2c7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:04.206568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759ad98-778c-4246-8ca3-f57d163fa2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:04.206664 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759ad98-778c-4246-8ca3-f57d163fa2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:04.227198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7759ad98-778c-4246-8ca3-f57d163fa2c7 None None] MFA Rules not processed for user `474c468a610140f8a5b18d5a1ed1d18e`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:04.228425 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7759ad98-778c-4246-8ca3-f57d163fa2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:04.238404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 683/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Jun 25 15:59:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:04.527496 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=72 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:59:05.368517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c887d27-ec02-47b8-81cc-8a4451278584 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:05.369160 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c887d27-ec02-47b8-81cc-8a4451278584 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:05.369160 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c887d27-ec02-47b8-81cc-8a4451278584 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:05.370368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 671/1354] 10.222.0.13 () {54 vars in 921 bytes} [Sat Jun 25 15:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:05.376305 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c401ae-1e41-45f5-ad42-5772b2475aad None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:05.376788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c401ae-1e41-45f5-ad42-5772b2475aad None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:05.377000 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c401ae-1e41-45f5-ad42-5772b2475aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:05.396518 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-55c401ae-1e41-45f5-ad42-5772b2475aad None None] MFA Rules not processed for user `474c468a610140f8a5b18d5a1ed1d18e`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:05.397559 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-55c401ae-1e41-45f5-ad42-5772b2475aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:05.407785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 684/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Jun 25 15:59:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:09.007130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc784c4-f229-478a-87a0-0112b4fc9fd4 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:09.007679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc784c4-f229-478a-87a0-0112b4fc9fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:09.007907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc784c4-f229-478a-87a0-0112b4fc9fd4 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:09.010025 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 672/1356] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:59:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:09.016435 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32640e35-2981-41d0-ae5f-d45025df2e0c None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:09.016930 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32640e35-2981-41d0-ae5f-d45025df2e0c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:09.016930 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32640e35-2981-41d0-ae5f-d45025df2e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:09.037728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-32640e35-2981-41d0-ae5f-d45025df2e0c None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:09.039159 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-32640e35-2981-41d0-ae5f-d45025df2e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:09.052760 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 685/1357] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:09.103178 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a70224-4e73-4f1a-bd0d-73be4976cd81 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:09.104533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a70224-4e73-4f1a-bd0d-73be4976cd81 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:09.105154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a70224-4e73-4f1a-bd0d-73be4976cd81 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:09.107672 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 673/1358] 10.222.0.13 () {54 vars in 961 bytes} [Sat Jun 25 15:59:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:09.117238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976209ee-117f-4263-9b8c-abeceaca8d72 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:09.118250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976209ee-117f-4263-9b8c-abeceaca8d72 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:09.118830 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976209ee-117f-4263-9b8c-abeceaca8d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:09.174299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-976209ee-117f-4263-9b8c-abeceaca8d72 None None] MFA Rules not processed for user `3622c5ccb7534e7c8e8342b4f6643bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:09.175676 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-976209ee-117f-4263-9b8c-abeceaca8d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:09.260669 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 686/1359] 10.222.0.13 () {58 vars in 1058 bytes} [Sat Jun 25 15:59:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 15:59:09.269317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a70224-4e73-4f1a-bd0d-73be4976cd81 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:59:09.284159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a70224-4e73-4f1a-bd0d-73be4976cd81 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:59:09.289747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193a55c-a387-481d-ad1a-3f3af8123cbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:59:09.290487 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193a55c-a387-481d-ad1a-3f3af8123cbd service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:09.290826 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193a55c-a387-481d-ad1a-3f3af8123cbd service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:09.291146 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193a55c-a387-481d-ad1a-3f3af8123cbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:09.301253 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193a55c-a387-481d-ad1a-3f3af8123cbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:59:09.302961 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193a55c-a387-481d-ad1a-3f3af8123cbd service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:59:09.313309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 674/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:59:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 15:59:11.545077 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc2efe-e04b-4fd2-bc35-eb388b8f8372 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:11.545562 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc2efe-e04b-4fd2-bc35-eb388b8f8372 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:11.545663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc2efe-e04b-4fd2-bc35-eb388b8f8372 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:11.548762 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 687/1361] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:59:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:11.553225 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a1889-920f-4c84-8635-69f8652e35d7 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:11.553721 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a1889-920f-4c84-8635-69f8652e35d7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:11.554877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a1889-920f-4c84-8635-69f8652e35d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:11.575157 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-460a1889-920f-4c84-8635-69f8652e35d7 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:11.576483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-460a1889-920f-4c84-8635-69f8652e35d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:11.586521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 675/1362] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:11.632307 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc2efe-e04b-4fd2-bc35-eb388b8f8372 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:59:11.647248 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc2efe-e04b-4fd2-bc35-eb388b8f8372 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:59:11.652865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fadc8f-736f-4992-a4f1-3ccce3f25fb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:59:11.653509 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fadc8f-736f-4992-a4f1-3ccce3f25fb6 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:11.653787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fadc8f-736f-4992-a4f1-3ccce3f25fb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:11.654060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fadc8f-736f-4992-a4f1-3ccce3f25fb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:11.662480 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fadc8f-736f-4992-a4f1-3ccce3f25fb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:59:11.664054 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fadc8f-736f-4992-a4f1-3ccce3f25fb6 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:59:11.675703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 688/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:59:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:59:14.004901 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb827bd-f772-4412-a293-7f535dc45998 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:14.005373 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb827bd-f772-4412-a293-7f535dc45998 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:14.005465 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb827bd-f772-4412-a293-7f535dc45998 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:14.006833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 676/1364] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:14.012233 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d067edc-236c-494d-9717-5b5ff3604c0a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:14.012693 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d067edc-236c-494d-9717-5b5ff3604c0a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:14.012815 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d067edc-236c-494d-9717-5b5ff3604c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:14.031757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-2d067edc-236c-494d-9717-5b5ff3604c0a None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:14.032993 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2d067edc-236c-494d-9717-5b5ff3604c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:14.042712 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 689/1365] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:59:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:14.090593 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f60d78-d37c-42c4-87ca-9a955ce2bd40 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:14.090875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f60d78-d37c-42c4-87ca-9a955ce2bd40 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:14.091153 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f60d78-d37c-42c4-87ca-9a955ce2bd40 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:14.093341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 677/1366] 10.222.0.13 () {54 vars in 961 bytes} [Sat Jun 25 15:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:14.099687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03471a-c6c9-4def-9d24-e17cf5b835d3 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:14.099899 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03471a-c6c9-4def-9d24-e17cf5b835d3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:14.100178 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03471a-c6c9-4def-9d24-e17cf5b835d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:14.121715 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8b03471a-c6c9-4def-9d24-e17cf5b835d3 None None] MFA Rules not processed for user `3622c5ccb7534e7c8e8342b4f6643bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:14.122843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8b03471a-c6c9-4def-9d24-e17cf5b835d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:14.133414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 690/1367] 10.222.0.13 () {58 vars in 1058 bytes} [Sat Jun 25 15:59:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:14.142483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f60d78-d37c-42c4-87ca-9a955ce2bd40 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:59:14.156945 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f60d78-d37c-42c4-87ca-9a955ce2bd40 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:59:14.163031 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d93c4-c6d7-403d-b585-b21b2bdda5cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:59:14.163671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d93c4-c6d7-403d-b585-b21b2bdda5cb service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:14.163954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d93c4-c6d7-403d-b585-b21b2bdda5cb service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:14.164395 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d93c4-c6d7-403d-b585-b21b2bdda5cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:14.173230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d93c4-c6d7-403d-b585-b21b2bdda5cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:59:14.174863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d93c4-c6d7-403d-b585-b21b2bdda5cb service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:59:14.184804 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 678/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:59:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:59:16.277565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a914d1e-f78d-4a7e-8106-72e940ecf778 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:16.278159 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a914d1e-f78d-4a7e-8106-72e940ecf778 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:16.278159 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a914d1e-f78d-4a7e-8106-72e940ecf778 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:16.279685 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 691/1369] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:16.285418 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb1f9a-12b5-4980-bcce-d454d6588de8 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:16.285836 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb1f9a-12b5-4980-bcce-d454d6588de8 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:16.285958 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb1f9a-12b5-4980-bcce-d454d6588de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:16.305454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-04fb1f9a-12b5-4980-bcce-d454d6588de8 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:16.307024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-04fb1f9a-12b5-4980-bcce-d454d6588de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:16.315989 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 679/1370] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:59:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:16.360218 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a914d1e-f78d-4a7e-8106-72e940ecf778 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:59:16.376588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a914d1e-f78d-4a7e-8106-72e940ecf778 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:59:16.382899 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01b79f-e18f-4206-94b2-1fdf3459a83e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:59:16.383599 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01b79f-e18f-4206-94b2-1fdf3459a83e service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:16.383894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01b79f-e18f-4206-94b2-1fdf3459a83e service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:16.384169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01b79f-e18f-4206-94b2-1fdf3459a83e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:16.394432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01b79f-e18f-4206-94b2-1fdf3459a83e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:59:16.396070 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01b79f-e18f-4206-94b2-1fdf3459a83e service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:59:16.407548 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 692/1371] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:59:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:59:17.096071 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a98c5-ce2e-402c-9b75-a1b4ba206b30 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:17.096310 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a98c5-ce2e-402c-9b75-a1b4ba206b30 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:17.096660 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a98c5-ce2e-402c-9b75-a1b4ba206b30 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:17.098213 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 680/1372] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:17.103678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e723f4-9d1a-4f2f-b9bb-393b50d884e0 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:17.103938 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e723f4-9d1a-4f2f-b9bb-393b50d884e0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:17.104449 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e723f4-9d1a-4f2f-b9bb-393b50d884e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:17.123353 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a0e723f4-9d1a-4f2f-b9bb-393b50d884e0 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:17.124510 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a0e723f4-9d1a-4f2f-b9bb-393b50d884e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:17.133364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 693/1373] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:59:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:17.439790 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7018f-c41c-4c42-9c2a-a5ba77bb3478 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:17.440227 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7018f-c41c-4c42-9c2a-a5ba77bb3478 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:17.440332 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7018f-c41c-4c42-9c2a-a5ba77bb3478 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:17.441996 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 681/1374] 10.222.0.13 () {54 vars in 963 bytes} [Sat Jun 25 15:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:17.447246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d14756-9a9c-4ab2-b591-663b8e51ee7f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:17.447710 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d14756-9a9c-4ab2-b591-663b8e51ee7f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:17.447793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d14756-9a9c-4ab2-b591-663b8e51ee7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:17.456004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:59:17.503658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b3d14756-9a9c-4ab2-b591-663b8e51ee7f None None] MFA Rules not processed for user `b00152c72a1f4755b09c1dbd1d874b11`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:17.505023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b3d14756-9a9c-4ab2-b591-663b8e51ee7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:17.572693 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 694/1375] 10.222.0.13 () {58 vars in 1060 bytes} [Sat Jun 25 15:59:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:17.580711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7018f-c41c-4c42-9c2a-a5ba77bb3478 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:59:17.591571 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7018f-c41c-4c42-9c2a-a5ba77bb3478 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:59:17.595154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd60a8-1f0a-432c-9798-f48e7d844e6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:59:17.595824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd60a8-1f0a-432c-9798-f48e7d844e6d service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:17.596075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd60a8-1f0a-432c-9798-f48e7d844e6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:17.596345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd60a8-1f0a-432c-9798-f48e7d844e6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:17.605758 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd60a8-1f0a-432c-9798-f48e7d844e6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:59:17.607217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd60a8-1f0a-432c-9798-f48e7d844e6d service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:59:17.617910 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 682/1376] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:59:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:59:18.458039 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9765f8-e24e-4a05-885a-ac5bc443c1a7 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:18.458869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9765f8-e24e-4a05-885a-ac5bc443c1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:18.458869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9765f8-e24e-4a05-885a-ac5bc443c1a7 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:18.463635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 695/1377] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:59:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:18.467772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba34f9e-a23b-4d94-842a-03e791332ac3 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:18.468866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba34f9e-a23b-4d94-842a-03e791332ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:18.469320 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba34f9e-a23b-4d94-842a-03e791332ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:18.499516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-eba34f9e-a23b-4d94-842a-03e791332ac3 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:18.500836 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-eba34f9e-a23b-4d94-842a-03e791332ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:18.511036 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 683/1378] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:18.555426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9765f8-e24e-4a05-885a-ac5bc443c1a7 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:59:18.571346 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9765f8-e24e-4a05-885a-ac5bc443c1a7 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:59:18.577788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a4d18-bb53-43af-b155-815059826dee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:59:18.578578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a4d18-bb53-43af-b155-815059826dee service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:18.578833 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a4d18-bb53-43af-b155-815059826dee service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:18.579140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a4d18-bb53-43af-b155-815059826dee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:18.588356 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a4d18-bb53-43af-b155-815059826dee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:59:18.589999 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a4d18-bb53-43af-b155-815059826dee service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:59:18.600356 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 696/1379] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:59:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:59:20.668502 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0615638-19f8-4a05-8980-3b9ab6637eac None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:20.669098 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0615638-19f8-4a05-8980-3b9ab6637eac None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:20.669098 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0615638-19f8-4a05-8980-3b9ab6637eac None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:20.670749 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 684/1380] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:20.676891 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e90da-ac81-4d71-acde-2360093816b1 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:20.677366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e90da-ac81-4d71-acde-2360093816b1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:20.677624 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e90da-ac81-4d71-acde-2360093816b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:20.696109 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c40e90da-ac81-4d71-acde-2360093816b1 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:20.697349 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c40e90da-ac81-4d71-acde-2360093816b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:20.705943 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 697/1381] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:20.753584 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0615638-19f8-4a05-8980-3b9ab6637eac None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:59:20.770198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0615638-19f8-4a05-8980-3b9ab6637eac None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:59:20.775704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97a59f-ec60-4e13-a89f-936cb1748513 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:59:20.776486 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97a59f-ec60-4e13-a89f-936cb1748513 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:20.776872 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97a59f-ec60-4e13-a89f-936cb1748513 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:20.777308 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97a59f-ec60-4e13-a89f-936cb1748513 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:20.786251 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97a59f-ec60-4e13-a89f-936cb1748513 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:59:20.788137 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97a59f-ec60-4e13-a89f-936cb1748513 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:59:20.798640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 685/1382] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:59:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:59:22.866661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b39ee4-0322-4a86-a2f1-d614609e22de None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:22.867122 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b39ee4-0322-4a86-a2f1-d614609e22de None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:22.867235 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b39ee4-0322-4a86-a2f1-d614609e22de None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:22.870059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 698/1383] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:59:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:22.873970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9e298-0786-4211-bce4-131df8f08124 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:22.874511 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9e298-0786-4211-bce4-131df8f08124 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:22.875219 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9e298-0786-4211-bce4-131df8f08124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:22.894555 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-7ff9e298-0786-4211-bce4-131df8f08124 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:22.895802 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7ff9e298-0786-4211-bce4-131df8f08124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:22.904531 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 686/1384] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:22.947201 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b39ee4-0322-4a86-a2f1-d614609e22de None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:59:22.963266 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b39ee4-0322-4a86-a2f1-d614609e22de None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:59:22.970056 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac3e2a-26c2-4623-92f2-de41d9ba08ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:59:22.970789 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac3e2a-26c2-4623-92f2-de41d9ba08ce service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:22.971140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac3e2a-26c2-4623-92f2-de41d9ba08ce service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:22.971483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac3e2a-26c2-4623-92f2-de41d9ba08ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:22.981261 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac3e2a-26c2-4623-92f2-de41d9ba08ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:59:22.982931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac3e2a-26c2-4623-92f2-de41d9ba08ce service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:59:22.993873 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 699/1385] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:59:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:59:25.044911 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad04528-da9b-4fde-a9c6-1bb57de6f794 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:25.045386 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad04528-da9b-4fde-a9c6-1bb57de6f794 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:25.045484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad04528-da9b-4fde-a9c6-1bb57de6f794 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:25.046849 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 687/1386] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:25.052156 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88478fa7-8b9e-42f2-bd08-4f731d7fc2bd None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:25.052634 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88478fa7-8b9e-42f2-bd08-4f731d7fc2bd None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:25.052718 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88478fa7-8b9e-42f2-bd08-4f731d7fc2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:25.071884 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-88478fa7-8b9e-42f2-bd08-4f731d7fc2bd None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:25.073150 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-88478fa7-8b9e-42f2-bd08-4f731d7fc2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:25.101991 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 700/1387] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:25.134935 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad04528-da9b-4fde-a9c6-1bb57de6f794 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:59:25.171392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad04528-da9b-4fde-a9c6-1bb57de6f794 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:59:25.179946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c267b2-c4b2-462c-8228-0e6e03b3ead0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:59:25.180923 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c267b2-c4b2-462c-8228-0e6e03b3ead0 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:25.181330 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c267b2-c4b2-462c-8228-0e6e03b3ead0 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:25.181733 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c267b2-c4b2-462c-8228-0e6e03b3ead0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:25.192978 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c267b2-c4b2-462c-8228-0e6e03b3ead0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:59:25.194637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c267b2-c4b2-462c-8228-0e6e03b3ead0 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:59:25.205408 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 688/1388] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:59:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:59:27.341033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80c30d-91fd-410e-9c1f-6c80f3bb673d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:27.341794 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80c30d-91fd-410e-9c1f-6c80f3bb673d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:27.341794 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80c30d-91fd-410e-9c1f-6c80f3bb673d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:27.343468 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 701/1389] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:59:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:27.349333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772c8bc-6278-4a7c-b875-5d37f4441c06 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:27.349840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772c8bc-6278-4a7c-b875-5d37f4441c06 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:27.349944 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772c8bc-6278-4a7c-b875-5d37f4441c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:27.370175 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-8772c8bc-6278-4a7c-b875-5d37f4441c06 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:27.371443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8772c8bc-6278-4a7c-b875-5d37f4441c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:27.380629 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 689/1390] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:27.435770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80c30d-91fd-410e-9c1f-6c80f3bb673d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:59:27.452365 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80c30d-91fd-410e-9c1f-6c80f3bb673d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:59:27.458462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee40ded-35f1-4017-b220-ed5b7077cb48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:59:27.459135 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee40ded-35f1-4017-b220-ed5b7077cb48 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:27.459434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee40ded-35f1-4017-b220-ed5b7077cb48 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:27.459725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee40ded-35f1-4017-b220-ed5b7077cb48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:27.468944 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee40ded-35f1-4017-b220-ed5b7077cb48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:59:27.470616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee40ded-35f1-4017-b220-ed5b7077cb48 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:59:27.480977 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 702/1391] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:59:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:59:29.592951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8711b-c79a-4e6b-b2db-d3d898e46e19 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:29.593773 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8711b-c79a-4e6b-b2db-d3d898e46e19 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:29.593773 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8711b-c79a-4e6b-b2db-d3d898e46e19 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:29.595000 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 690/1392] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:59:29.600658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76001ea4-7231-4896-b45f-8e41b9651882 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:29.601101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76001ea4-7231-4896-b45f-8e41b9651882 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:29.601243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76001ea4-7231-4896-b45f-8e41b9651882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:29.622652 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-76001ea4-7231-4896-b45f-8e41b9651882 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:29.623935 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-76001ea4-7231-4896-b45f-8e41b9651882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:29.633011 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 703/1393] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:29.678509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8711b-c79a-4e6b-b2db-d3d898e46e19 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:59:29.697499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8711b-c79a-4e6b-b2db-d3d898e46e19 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:59:29.705184 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79587ec7-5d92-4229-b326-e0b01404b46c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:59:29.706077 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79587ec7-5d92-4229-b326-e0b01404b46c service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:29.706426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79587ec7-5d92-4229-b326-e0b01404b46c service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:29.706771 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79587ec7-5d92-4229-b326-e0b01404b46c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:29.717759 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79587ec7-5d92-4229-b326-e0b01404b46c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:59:29.720157 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79587ec7-5d92-4229-b326-e0b01404b46c service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:59:29.732680 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 691/1394] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:59:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:59:31.809768 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf2d77-4842-4fc2-b82e-dbf01aaf4089 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:31.810340 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf2d77-4842-4fc2-b82e-dbf01aaf4089 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:31.810340 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf2d77-4842-4fc2-b82e-dbf01aaf4089 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:31.811784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 704/1395] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:31.817510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ea83b-9d8c-4506-8b46-c61571072629 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:31.817968 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ea83b-9d8c-4506-8b46-c61571072629 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:31.818064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ea83b-9d8c-4506-8b46-c61571072629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:31.837058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-964ea83b-9d8c-4506-8b46-c61571072629 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:31.838256 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-964ea83b-9d8c-4506-8b46-c61571072629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:31.846509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 692/1396] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:31.892035 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf2d77-4842-4fc2-b82e-dbf01aaf4089 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:59:31.907138 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf2d77-4842-4fc2-b82e-dbf01aaf4089 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:59:31.914076 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8a865-ad67-499c-867f-8cfc6dd64423 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:59:31.914796 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8a865-ad67-499c-867f-8cfc6dd64423 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:31.915250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8a865-ad67-499c-867f-8cfc6dd64423 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:31.915560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8a865-ad67-499c-867f-8cfc6dd64423 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:31.925587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8a865-ad67-499c-867f-8cfc6dd64423 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:59:31.927186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8a865-ad67-499c-867f-8cfc6dd64423 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:59:31.937989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 705/1397] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:59:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:59:32.831739 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48aece-5bf2-4f18-a728-62ce5e622246 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:32.832289 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48aece-5bf2-4f18-a728-62ce5e622246 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:32.832289 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48aece-5bf2-4f18-a728-62ce5e622246 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:32.834403 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 693/1398] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:59:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:32.843371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53b0b6-3deb-43b8-bdc0-596745d4ea8d None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:32.843887 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53b0b6-3deb-43b8-bdc0-596745d4ea8d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:32.843887 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53b0b6-3deb-43b8-bdc0-596745d4ea8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:32.865286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-fd53b0b6-3deb-43b8-bdc0-596745d4ea8d None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:32.866554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-fd53b0b6-3deb-43b8-bdc0-596745d4ea8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:32.875044 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 706/1399] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:59:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:32.945363 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8786b65-d43a-488b-938d-8d7af9cf6a73 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:32.945777 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8786b65-d43a-488b-938d-8d7af9cf6a73 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:32.946073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8786b65-d43a-488b-938d-8d7af9cf6a73 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:32.947529 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 694/1400] 10.222.0.13 () {54 vars in 962 bytes} [Sat Jun 25 15:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:32.953314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9fc1b-1762-43ba-bee4-c72765d40649 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:32.953681 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9fc1b-1762-43ba-bee4-c72765d40649 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:32.954023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9fc1b-1762-43ba-bee4-c72765d40649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:33.012346 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-52b9fc1b-1762-43ba-bee4-c72765d40649 None None] MFA Rules not processed for user `5cba086728824ec68fb0ee90fb3f99dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:33.013814 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-52b9fc1b-1762-43ba-bee4-c72765d40649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:33.054376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:59:33.090115 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 707/1401] 10.222.0.13 () {58 vars in 1059 bytes} [Sat Jun 25 15:59:32 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:33.098596 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8786b65-d43a-488b-938d-8d7af9cf6a73 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:59:33.120530 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8786b65-d43a-488b-938d-8d7af9cf6a73 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:59:33.127210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384a3c0-769b-4ab8-9779-13a1e99a5df9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cba086728824ec68fb0ee90fb3f99dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:59:33.127880 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384a3c0-769b-4ab8-9779-13a1e99a5df9 service glance] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:33.128160 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384a3c0-769b-4ab8-9779-13a1e99a5df9 service glance] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:33.128463 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384a3c0-769b-4ab8-9779-13a1e99a5df9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:33.144284 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384a3c0-769b-4ab8-9779-13a1e99a5df9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:59:33.147367 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384a3c0-769b-4ab8-9779-13a1e99a5df9 service glance] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:59:33.159580 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 695/1402] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:59:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 15:59:33.952305 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672c249-9b8a-4137-bc20-ffd945fe1ec0 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:33.952915 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672c249-9b8a-4137-bc20-ffd945fe1ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:33.953029 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672c249-9b8a-4137-bc20-ffd945fe1ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:34.004998 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-6672c249-9b8a-4137-bc20-ffd945fe1ec0 None None] MFA Rules not processed for user `28b3e73439e64050b406a9f3e9b0160a`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:34.006212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6672c249-9b8a-4137-bc20-ffd945fe1ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:34.074415 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 708/1403] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Jun 25 15:59:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:34.099412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3cd4e-2d32-4af1-a61f-e910d7d12d25 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:34.099877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3cd4e-2d32-4af1-a61f-e910d7d12d25 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:34.099969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3cd4e-2d32-4af1-a61f-e910d7d12d25 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:34.101558 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 696/1404] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 15:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:59:34.107029 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f0f0a-6937-4fd8-a9c8-73823730bb2b None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:34.107386 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f0f0a-6937-4fd8-a9c8-73823730bb2b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:34.107386 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f0f0a-6937-4fd8-a9c8-73823730bb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:34.119453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dfb6a-f48b-4f30-8914-4962a2ebea75 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:34.119734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dfb6a-f48b-4f30-8914-4962a2ebea75 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:34.120011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dfb6a-f48b-4f30-8914-4962a2ebea75 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:34.121451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 697/1405] 10.222.0.13 () {54 vars in 942 bytes} [Sat Jun 25 15:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:34.125916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-310f0f0a-6937-4fd8-a9c8-73823730bb2b None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:34.127129 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-310f0f0a-6937-4fd8-a9c8-73823730bb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:34.128374 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c319c-d794-418c-9cce-dd1d33fbd393 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:34.128678 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c319c-d794-418c-9cce-dd1d33fbd393 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:34.128953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c319c-d794-418c-9cce-dd1d33fbd393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:34.135107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 709/1406] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 15:59:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:34.177656 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ca9c319c-d794-418c-9cce-dd1d33fbd393 None None] MFA Rules not processed for user `19788c4239ca44f9a31a7d32086c8947`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:34.178852 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ca9c319c-d794-418c-9cce-dd1d33fbd393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:34.179727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f0f0a-6937-4fd8-a9c8-73823730bb2b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:59:34.192121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f0f0a-6937-4fd8-a9c8-73823730bb2b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:59:34.197524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf57aa-194e-4906-920a-5f5d67b6e613 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:59:34.198176 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf57aa-194e-4906-920a-5f5d67b6e613 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:34.198460 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf57aa-194e-4906-920a-5f5d67b6e613 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:34.198780 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf57aa-194e-4906-920a-5f5d67b6e613 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:34.207464 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf57aa-194e-4906-920a-5f5d67b6e613 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:59:34.209101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf57aa-194e-4906-920a-5f5d67b6e613 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:59:34.218804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 710/1407] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:59:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 15:59:34.232850 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:59:34.252772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 698/1408] 10.222.0.13 () {58 vars in 1039 bytes} [Sat Jun 25 15:59:34 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:34.262304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf57aa-194e-4906-920a-5f5d67b6e613 service nova] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:59:34.274733 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf57aa-194e-4906-920a-5f5d67b6e613 service nova] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:59:34.280425 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834cc410-e8d7-49a6-bc99-f1c59bd65bd3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19788c4239ca44f9a31a7d32086c8947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:59:34.281210 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cc410-e8d7-49a6-bc99-f1c59bd65bd3 service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:34.281569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cc410-e8d7-49a6-bc99-f1c59bd65bd3 service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:34.281881 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cc410-e8d7-49a6-bc99-f1c59bd65bd3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:34.290773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834cc410-e8d7-49a6-bc99-f1c59bd65bd3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:59:34.292515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834cc410-e8d7-49a6-bc99-f1c59bd65bd3 service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:59:34.302090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 711/1409] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Jun 25 15:59:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 25 15:59:44.249491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:59:44.298117 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=65 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 15:59:56.444028 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af24cd2-e66f-4922-817e-140cd2e40f6a None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:56.444608 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af24cd2-e66f-4922-817e-140cd2e40f6a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:56.444608 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af24cd2-e66f-4922-817e-140cd2e40f6a None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:56.445964 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 699/1410] 10.222.0.13 () {54 vars in 918 bytes} [Sat Jun 25 15:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:56.451300 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c0a32-1e9d-425d-8e15-397a7d91419b None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:56.451725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c0a32-1e9d-425d-8e15-397a7d91419b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:56.451835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c0a32-1e9d-425d-8e15-397a7d91419b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:56.471123 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-234c0a32-1e9d-425d-8e15-397a7d91419b None None] MFA Rules not processed for user `474c468a610140f8a5b18d5a1ed1d18e`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:56.472320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-234c0a32-1e9d-425d-8e15-397a7d91419b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:56.481615 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 712/1411] 10.222.0.13 () {58 vars in 1015 bytes} [Sat Jun 25 15:59:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:56.495839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03753875-9f12-426b-a12f-b0510aec7004 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:56.496124 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03753875-9f12-426b-a12f-b0510aec7004 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:56.496544 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03753875-9f12-426b-a12f-b0510aec7004 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:56.497912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 700/1412] 10.222.0.13 () {54 vars in 918 bytes} [Sat Jun 25 15:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 15:59:56.502832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7456af-a4d9-46fc-b806-2dec6fe6ac33 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:56.503124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7456af-a4d9-46fc-b806-2dec6fe6ac33 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:56.503415 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7456af-a4d9-46fc-b806-2dec6fe6ac33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:56.523040 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-4c7456af-a4d9-46fc-b806-2dec6fe6ac33 None None] MFA Rules not processed for user `3622c5ccb7534e7c8e8342b4f6643bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:56.524229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4c7456af-a4d9-46fc-b806-2dec6fe6ac33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:56.534259 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 713/1413] 10.222.0.13 () {58 vars in 1015 bytes} [Sat Jun 25 15:59:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:59.048364 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d31c66-ea52-4735-9d85-a6981abd571f None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:59.048947 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d31c66-ea52-4735-9d85-a6981abd571f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:59.048947 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d31c66-ea52-4735-9d85-a6981abd571f None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:59.050192 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 25 15:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 15:59:59.055708 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1b27c-b292-4d26-85e7-9ad1059e91cc None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:59.056174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1b27c-b292-4d26-85e7-9ad1059e91cc None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:59.056275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1b27c-b292-4d26-85e7-9ad1059e91cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:59.076327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-08a1b27c-b292-4d26-85e7-9ad1059e91cc None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 15:59:59.077645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-08a1b27c-b292-4d26-85e7-9ad1059e91cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 15:59:59.086019 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 25 15:59:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 15:59:59.132121 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d31c66-ea52-4735-9d85-a6981abd571f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 15:59:59.152009 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d31c66-ea52-4735-9d85-a6981abd571f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 15:59:59.158946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b853e-48c5-4487-8925-25e54877b4d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 15:59:59.159587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b853e-48c5-4487-8925-25e54877b4d7 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 15:59:59.159864 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b853e-48c5-4487-8925-25e54877b4d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 15:59:59.160131 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b853e-48c5-4487-8925-25e54877b4d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 15:59:59.168391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b853e-48c5-4487-8925-25e54877b4d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 15:59:59.170027 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b853e-48c5-4487-8925-25e54877b4d7 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 15:59:59.179712 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 702/1416] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 15:59:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:00.079564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f8e7b-2fe9-490d-9813-e13a9c793766 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:00.080459 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f8e7b-2fe9-490d-9813-e13a9c793766 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:00.080459 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f8e7b-2fe9-490d-9813-e13a9c793766 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:00.082631 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 25 16:00:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:00.087773 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394b6fb-31b3-4ab7-b913-7087564a50fa None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:00.088215 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394b6fb-31b3-4ab7-b913-7087564a50fa None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:00.088320 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394b6fb-31b3-4ab7-b913-7087564a50fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:00.108098 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-9394b6fb-31b3-4ab7-b913-7087564a50fa None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:00.109343 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9394b6fb-31b3-4ab7-b913-7087564a50fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:00.117956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 25 16:00:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:00.271046 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f8e7b-2fe9-490d-9813-e13a9c793766 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:00.291573 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f8e7b-2fe9-490d-9813-e13a9c793766 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:00.298545 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d801fa1-fc64-4605-a2f6-da2c331a65e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:00.299194 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d801fa1-fc64-4605-a2f6-da2c331a65e1 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:00.299487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d801fa1-fc64-4605-a2f6-da2c331a65e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:00.299759 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d801fa1-fc64-4605-a2f6-da2c331a65e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:00.308155 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d801fa1-fc64-4605-a2f6-da2c331a65e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:00.309780 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d801fa1-fc64-4605-a2f6-da2c331a65e1 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:00.319899 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 716/1419] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:01.200014 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4bd40-911e-437d-aea5-a2d217a28129 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:01.200635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4bd40-911e-437d-aea5-a2d217a28129 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:01.200635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4bd40-911e-437d-aea5-a2d217a28129 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:01.201971 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 25 16:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:01.208224 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b32b4-0d4d-4d71-9100-0e0edbce53ba None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:01.208948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b32b4-0d4d-4d71-9100-0e0edbce53ba None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:01.209183 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b32b4-0d4d-4d71-9100-0e0edbce53ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:01.238231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ea1b32b4-0d4d-4d71-9100-0e0edbce53ba None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:01.239450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ea1b32b4-0d4d-4d71-9100-0e0edbce53ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:01.249045 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 25 16:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:01.398798 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4bd40-911e-437d-aea5-a2d217a28129 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:01.424252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4bd40-911e-437d-aea5-a2d217a28129 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:01.435091 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117fd1b-9647-4920-8eb0-a21d5a249424 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:01.435707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117fd1b-9647-4920-8eb0-a21d5a249424 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:01.436013 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117fd1b-9647-4920-8eb0-a21d5a249424 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:01.436274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117fd1b-9647-4920-8eb0-a21d5a249424 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:01.445815 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117fd1b-9647-4920-8eb0-a21d5a249424 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:01.447648 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117fd1b-9647-4920-8eb0-a21d5a249424 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:01.461233 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 705/1422] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:02.312050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62f306-860c-4e41-bd48-468d44826e5e None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:02.312837 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62f306-860c-4e41-bd48-468d44826e5e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:02.312956 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62f306-860c-4e41-bd48-468d44826e5e None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:02.314468 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 25 16:00:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:02.320694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae63ec6-d198-4d42-b9c2-a517812c6a76 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:02.321157 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae63ec6-d198-4d42-b9c2-a517812c6a76 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:02.321267 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae63ec6-d198-4d42-b9c2-a517812c6a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:02.348682 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-4ae63ec6-d198-4d42-b9c2-a517812c6a76 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:02.349896 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4ae63ec6-d198-4d42-b9c2-a517812c6a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:02.359053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 25 16:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:02.510270 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62f306-860c-4e41-bd48-468d44826e5e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:02.533992 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62f306-860c-4e41-bd48-468d44826e5e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:02.540527 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af9cfc2-4303-4b8b-811b-38d02b6905af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:02.541174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9cfc2-4303-4b8b-811b-38d02b6905af service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:02.541449 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9cfc2-4303-4b8b-811b-38d02b6905af service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:02.541717 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9cfc2-4303-4b8b-811b-38d02b6905af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:02.550736 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af9cfc2-4303-4b8b-811b-38d02b6905af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:02.552314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af9cfc2-4303-4b8b-811b-38d02b6905af service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:02.561632 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 719/1425] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:03.445862 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27966ac1-3d02-47d6-8608-1ab0772e8ea6 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:03.446358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27966ac1-3d02-47d6-8608-1ab0772e8ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:03.446358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27966ac1-3d02-47d6-8608-1ab0772e8ea6 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:03.447794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 25 16:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:03.456091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7d40e-2393-4c20-b336-2667fbb5d7a3 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:03.456588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7d40e-2393-4c20-b336-2667fbb5d7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:03.456695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7d40e-2393-4c20-b336-2667fbb5d7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:03.477361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-21d7d40e-2393-4c20-b336-2667fbb5d7a3 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:03.478570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-21d7d40e-2393-4c20-b336-2667fbb5d7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:03.489203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 25 16:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:03.641478 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27966ac1-3d02-47d6-8608-1ab0772e8ea6 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:03.667152 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27966ac1-3d02-47d6-8608-1ab0772e8ea6 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:03.675385 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44017c-e090-46d2-82dc-d93824d0dd29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:03.676036 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44017c-e090-46d2-82dc-d93824d0dd29 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:03.676448 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44017c-e090-46d2-82dc-d93824d0dd29 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:03.676727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44017c-e090-46d2-82dc-d93824d0dd29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:03.685295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44017c-e090-46d2-82dc-d93824d0dd29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:03.686835 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44017c-e090-46d2-82dc-d93824d0dd29 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:03.696556 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 708/1428] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:04.506190 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1cc2e-4a2c-4b48-9ce9-6ef8536f7447 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:04.506811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1cc2e-4a2c-4b48-9ce9-6ef8536f7447 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:04.506811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a1cc2e-4a2c-4b48-9ce9-6ef8536f7447 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:04.508959 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 25 16:00:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:04.514110 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd637e2-bd33-4b1a-ac5e-bf556e48c695 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:04.514574 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd637e2-bd33-4b1a-ac5e-bf556e48c695 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:04.514667 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd637e2-bd33-4b1a-ac5e-bf556e48c695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:04.533951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5bd637e2-bd33-4b1a-ac5e-bf556e48c695 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:04.535133 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5bd637e2-bd33-4b1a-ac5e-bf556e48c695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:04.543447 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 25 16:00:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:00:04.698613 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1cc2e-4a2c-4b48-9ce9-6ef8536f7447 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:04.721815 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a1cc2e-4a2c-4b48-9ce9-6ef8536f7447 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:04.728739 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68139031-0191-46cd-abd9-07d3eafb0768 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:04.729388 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68139031-0191-46cd-abd9-07d3eafb0768 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:04.729671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68139031-0191-46cd-abd9-07d3eafb0768 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:04.729949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68139031-0191-46cd-abd9-07d3eafb0768 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:04.739655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68139031-0191-46cd-abd9-07d3eafb0768 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:04.741290 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68139031-0191-46cd-abd9-07d3eafb0768 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:04.751906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 722/1431] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:05.575288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492e785-705c-4851-8496-830c1f35418a None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:05.575836 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492e785-705c-4851-8496-830c1f35418a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:05.575836 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492e785-705c-4851-8496-830c1f35418a None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:05.577957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 25 16:00:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:05.583538 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4e00d-49e2-4026-893a-9c88454b6aad None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:05.583981 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4e00d-49e2-4026-893a-9c88454b6aad None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:05.584077 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4e00d-49e2-4026-893a-9c88454b6aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:05.604992 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ede4e00d-49e2-4026-893a-9c88454b6aad None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:05.606222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ede4e00d-49e2-4026-893a-9c88454b6aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:05.615829 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 25 16:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:05.767698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492e785-705c-4851-8496-830c1f35418a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:05.789341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492e785-705c-4851-8496-830c1f35418a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:05.798471 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699adbe-9f97-4e60-bd10-c1058f750b80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:05.799113 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699adbe-9f97-4e60-bd10-c1058f750b80 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:05.799388 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699adbe-9f97-4e60-bd10-c1058f750b80 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:05.799653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699adbe-9f97-4e60-bd10-c1058f750b80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:05.808945 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699adbe-9f97-4e60-bd10-c1058f750b80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:05.810592 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699adbe-9f97-4e60-bd10-c1058f750b80 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:05.820738 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 711/1434] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:06.636007 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe3486-d4e3-40c1-9742-8166568b767d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:06.636958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe3486-d4e3-40c1-9742-8166568b767d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:06.636958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe3486-d4e3-40c1-9742-8166568b767d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:06.639108 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 25 16:00:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:06.645353 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4e555-e756-4044-9a9c-e3095a25786b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:06.645810 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4e555-e756-4044-9a9c-e3095a25786b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:06.645908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4e555-e756-4044-9a9c-e3095a25786b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:06.665294 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b7c4e555-e756-4044-9a9c-e3095a25786b None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:06.666514 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b7c4e555-e756-4044-9a9c-e3095a25786b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:06.676455 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 25 16:00:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:06.826684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe3486-d4e3-40c1-9742-8166568b767d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:06.844202 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe3486-d4e3-40c1-9742-8166568b767d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:06.853474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efdfdc-531f-4ef8-a67a-cb60ca793433 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:06.854118 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efdfdc-531f-4ef8-a67a-cb60ca793433 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:06.854395 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efdfdc-531f-4ef8-a67a-cb60ca793433 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:06.854668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efdfdc-531f-4ef8-a67a-cb60ca793433 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:06.863724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47efdfdc-531f-4ef8-a67a-cb60ca793433 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:06.865341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47efdfdc-531f-4ef8-a67a-cb60ca793433 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:06.875157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 725/1437] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:07.675471 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f4884-f6cb-4531-8054-7f754393e174 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:07.676448 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f4884-f6cb-4531-8054-7f754393e174 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:07.676711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f4884-f6cb-4531-8054-7f754393e174 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:07.679603 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 25 16:00:07 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:07.687843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763b263-c629-4996-8eca-1f98d4fb0940 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:07.688391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763b263-c629-4996-8eca-1f98d4fb0940 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:07.688566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763b263-c629-4996-8eca-1f98d4fb0940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:07.713158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3763b263-c629-4996-8eca-1f98d4fb0940 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:07.715052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3763b263-c629-4996-8eca-1f98d4fb0940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:07.725559 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 25 16:00:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:07.873058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f4884-f6cb-4531-8054-7f754393e174 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:07.888717 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f4884-f6cb-4531-8054-7f754393e174 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:07.895484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6070fe-0177-4b62-aefb-0c0a68deba04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:07.896114 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6070fe-0177-4b62-aefb-0c0a68deba04 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:07.896386 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6070fe-0177-4b62-aefb-0c0a68deba04 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:07.896679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6070fe-0177-4b62-aefb-0c0a68deba04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:07.905308 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6070fe-0177-4b62-aefb-0c0a68deba04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:07.906900 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6070fe-0177-4b62-aefb-0c0a68deba04 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:07.916367 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 714/1440] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:09.805469 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3932969-3da3-4ee5-8826-6d88ca5fe02b None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:09.806033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3932969-3da3-4ee5-8826-6d88ca5fe02b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:09.806033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3932969-3da3-4ee5-8826-6d88ca5fe02b None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:09.807411 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 25 16:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:09.812992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bbf8c-7eae-43a8-9eda-906d1506b8fc None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:09.813433 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bbf8c-7eae-43a8-9eda-906d1506b8fc None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:09.813538 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bbf8c-7eae-43a8-9eda-906d1506b8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:09.832291 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b57bbf8c-7eae-43a8-9eda-906d1506b8fc None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:09.833542 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b57bbf8c-7eae-43a8-9eda-906d1506b8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:09.843123 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 25 16:00:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:00:09.882388 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3932969-3da3-4ee5-8826-6d88ca5fe02b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:09.898516 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3932969-3da3-4ee5-8826-6d88ca5fe02b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:09.902938 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98807b-ee25-4dea-b27a-10222e52ce24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:09.903580 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98807b-ee25-4dea-b27a-10222e52ce24 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:09.903859 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98807b-ee25-4dea-b27a-10222e52ce24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:09.904137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98807b-ee25-4dea-b27a-10222e52ce24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:09.914240 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98807b-ee25-4dea-b27a-10222e52ce24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:09.915657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98807b-ee25-4dea-b27a-10222e52ce24 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:09.925383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 728/1443] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:12.100372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339f4d8-1752-43e1-bb61-1a0dbde7aab0 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:12.100988 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339f4d8-1752-43e1-bb61-1a0dbde7aab0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:12.100988 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339f4d8-1752-43e1-bb61-1a0dbde7aab0 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:12.102341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 25 16:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:12.108963 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc594f-2148-4a1a-809a-f1c69a02d335 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:12.109698 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc594f-2148-4a1a-809a-f1c69a02d335 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:12.109873 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc594f-2148-4a1a-809a-f1c69a02d335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:12.134745 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a5cc594f-2148-4a1a-809a-f1c69a02d335 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:12.135862 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a5cc594f-2148-4a1a-809a-f1c69a02d335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:12.150706 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 25 16:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:12.203098 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339f4d8-1752-43e1-bb61-1a0dbde7aab0 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:12.220344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339f4d8-1752-43e1-bb61-1a0dbde7aab0 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:12.224622 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd7826-7bfe-4d72-b475-f941885f1a8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:12.225269 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd7826-7bfe-4d72-b475-f941885f1a8b service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:12.225548 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd7826-7bfe-4d72-b475-f941885f1a8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:12.225819 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd7826-7bfe-4d72-b475-f941885f1a8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:12.235993 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd7826-7bfe-4d72-b475-f941885f1a8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:12.237514 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd7826-7bfe-4d72-b475-f941885f1a8b service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:12.247438 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 717/1446] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:00:14.607668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419eeb0-dece-4ece-aca8-69a77965aeaa None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:14.608614 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419eeb0-dece-4ece-aca8-69a77965aeaa None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:14.608614 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419eeb0-dece-4ece-aca8-69a77965aeaa None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:14.610192 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 25 16:00:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:14.616554 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946f394-c10e-4d23-a32b-c5e60ba0f0f6 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:14.616990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946f394-c10e-4d23-a32b-c5e60ba0f0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:14.617164 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946f394-c10e-4d23-a32b-c5e60ba0f0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:14.643632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d946f394-c10e-4d23-a32b-c5e60ba0f0f6 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:14.644877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d946f394-c10e-4d23-a32b-c5e60ba0f0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:14.653463 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 25 16:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:14.695550 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419eeb0-dece-4ece-aca8-69a77965aeaa None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:14.712902 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419eeb0-dece-4ece-aca8-69a77965aeaa None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:14.719970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0899b07-35db-4c70-ac6e-8489ed094498 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:14.720957 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0899b07-35db-4c70-ac6e-8489ed094498 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:14.721315 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0899b07-35db-4c70-ac6e-8489ed094498 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:14.721708 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0899b07-35db-4c70-ac6e-8489ed094498 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:14.736441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0899b07-35db-4c70-ac6e-8489ed094498 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:14.737940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0899b07-35db-4c70-ac6e-8489ed094498 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:14.747908 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 731/1449] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:16.881534 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bc64a-9784-4d2b-af4e-ce314b7c5b61 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:16.882020 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bc64a-9784-4d2b-af4e-ce314b7c5b61 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:16.882124 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bc64a-9784-4d2b-af4e-ce314b7c5b61 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:16.883440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 25 16:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:16.889280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dd284-48ef-4d93-b3d8-5c2271909440 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:16.889716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dd284-48ef-4d93-b3d8-5c2271909440 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:16.889846 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dd284-48ef-4d93-b3d8-5c2271909440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:16.909473 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8d7dd284-48ef-4d93-b3d8-5c2271909440 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:16.910687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8d7dd284-48ef-4d93-b3d8-5c2271909440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:16.919324 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 25 16:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:16.961124 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bc64a-9784-4d2b-af4e-ce314b7c5b61 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:16.974199 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bc64a-9784-4d2b-af4e-ce314b7c5b61 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:16.980136 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e63424-fa20-4bc0-b00c-11796a4456ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:16.981208 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e63424-fa20-4bc0-b00c-11796a4456ed service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:16.981678 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e63424-fa20-4bc0-b00c-11796a4456ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:16.982072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e63424-fa20-4bc0-b00c-11796a4456ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:16.993419 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e63424-fa20-4bc0-b00c-11796a4456ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:16.994914 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e63424-fa20-4bc0-b00c-11796a4456ed service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:17.005130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 720/1452] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:19.081654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8e486-cfd9-4a10-85c7-ff747089d001 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:19.082612 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8e486-cfd9-4a10-85c7-ff747089d001 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:19.082612 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8e486-cfd9-4a10-85c7-ff747089d001 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:19.084885 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 25 16:00:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:19.093832 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c3099-198b-4061-a73b-44a2a0903eea None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:19.094564 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c3099-198b-4061-a73b-44a2a0903eea None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:19.094715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c3099-198b-4061-a73b-44a2a0903eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:19.118560 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-405c3099-198b-4061-a73b-44a2a0903eea None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:19.119695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-405c3099-198b-4061-a73b-44a2a0903eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:19.128543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 25 16:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:19.170470 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8e486-cfd9-4a10-85c7-ff747089d001 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:19.182848 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8e486-cfd9-4a10-85c7-ff747089d001 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:19.186967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca77f7e-b30a-4e6f-bc79-f2aa7e37c9c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:19.187626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca77f7e-b30a-4e6f-bc79-f2aa7e37c9c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:19.187904 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca77f7e-b30a-4e6f-bc79-f2aa7e37c9c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:19.188176 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca77f7e-b30a-4e6f-bc79-f2aa7e37c9c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:19.198003 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca77f7e-b30a-4e6f-bc79-f2aa7e37c9c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:19.199459 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca77f7e-b30a-4e6f-bc79-f2aa7e37c9c3 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:19.209425 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 734/1455] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:21.283735 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d81d1-d401-4e46-8c46-dac926d8c9e6 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:21.284698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d81d1-d401-4e46-8c46-dac926d8c9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:21.284698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d81d1-d401-4e46-8c46-dac926d8c9e6 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:21.286857 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 25 16:00:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:21.295875 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ba6a9-a8a8-4f94-acee-da59e598c7f6 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:21.296355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ba6a9-a8a8-4f94-acee-da59e598c7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:21.297028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ba6a9-a8a8-4f94-acee-da59e598c7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:21.321211 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8e9ba6a9-a8a8-4f94-acee-da59e598c7f6 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:21.322424 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8e9ba6a9-a8a8-4f94-acee-da59e598c7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:21.332266 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 25 16:00:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:21.370253 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d81d1-d401-4e46-8c46-dac926d8c9e6 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:21.383189 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d81d1-d401-4e46-8c46-dac926d8c9e6 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:21.387772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37a3d0-60aa-4f15-9ff1-018209389fd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:21.388434 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37a3d0-60aa-4f15-9ff1-018209389fd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:21.388731 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37a3d0-60aa-4f15-9ff1-018209389fd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:21.389014 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37a3d0-60aa-4f15-9ff1-018209389fd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:21.399741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b37a3d0-60aa-4f15-9ff1-018209389fd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:21.401245 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b37a3d0-60aa-4f15-9ff1-018209389fd4 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:21.411493 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 723/1458] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:23.520478 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d6567-083d-4dd7-9d07-751c444f9162 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:23.521037 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d6567-083d-4dd7-9d07-751c444f9162 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:23.521037 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d6567-083d-4dd7-9d07-751c444f9162 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:23.523197 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 25 16:00:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:23.532334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c1aa2-0561-4b46-b093-f2575fc9f25d None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:23.532790 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c1aa2-0561-4b46-b093-f2575fc9f25d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:23.532999 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c1aa2-0561-4b46-b093-f2575fc9f25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:23.560611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-652c1aa2-0561-4b46-b093-f2575fc9f25d None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:23.562259 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-652c1aa2-0561-4b46-b093-f2575fc9f25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:23.572121 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 25 16:00:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:23.612383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d6567-083d-4dd7-9d07-751c444f9162 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:23.626543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d6567-083d-4dd7-9d07-751c444f9162 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:23.631271 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ba13fd-bf2c-41c5-a333-d771c96dc7ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:23.631957 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba13fd-bf2c-41c5-a333-d771c96dc7ae service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:23.632236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba13fd-bf2c-41c5-a333-d771c96dc7ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:23.632545 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba13fd-bf2c-41c5-a333-d771c96dc7ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:23.642801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba13fd-bf2c-41c5-a333-d771c96dc7ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:23.644247 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ba13fd-bf2c-41c5-a333-d771c96dc7ae service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:23.653970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 737/1461] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:25.721211 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34283f-1046-4ba9-b8de-e269be53702a None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:25.722064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34283f-1046-4ba9-b8de-e269be53702a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:25.722064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34283f-1046-4ba9-b8de-e269be53702a None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:25.724357 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 25 16:00:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:25.733188 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5812a-e1b1-4af0-8593-5c6300481adb None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:25.733922 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5812a-e1b1-4af0-8593-5c6300481adb None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:25.734205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5812a-e1b1-4af0-8593-5c6300481adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:25.745261 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=79 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:00:25.761762 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-2fb5812a-e1b1-4af0-8593-5c6300481adb None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:25.763011 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2fb5812a-e1b1-4af0-8593-5c6300481adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:25.775152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 25 16:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:25.815577 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34283f-1046-4ba9-b8de-e269be53702a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:25.823511 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=73 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:00:25.831447 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34283f-1046-4ba9-b8de-e269be53702a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:25.835422 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17f257-7a3e-466c-819f-4c1dd1d9c05a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:25.836068 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17f257-7a3e-466c-819f-4c1dd1d9c05a service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:25.836345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17f257-7a3e-466c-819f-4c1dd1d9c05a service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:25.836655 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17f257-7a3e-466c-819f-4c1dd1d9c05a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:25.846745 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17f257-7a3e-466c-819f-4c1dd1d9c05a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:25.848184 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17f257-7a3e-466c-819f-4c1dd1d9c05a service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:25.857866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 726/1464] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:27.944251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef51e5-3a82-4975-8516-70841b739dbd None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:27.944904 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef51e5-3a82-4975-8516-70841b739dbd None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:27.944904 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef51e5-3a82-4975-8516-70841b739dbd None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:27.946265 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 25 16:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:27.952292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681942d-2343-409d-ad84-54532e446d1d None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:27.952772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681942d-2343-409d-ad84-54532e446d1d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:27.952888 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681942d-2343-409d-ad84-54532e446d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:27.973396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-6681942d-2343-409d-ad84-54532e446d1d None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:27.974603 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6681942d-2343-409d-ad84-54532e446d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:27.983129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 25 16:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:28.023578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef51e5-3a82-4975-8516-70841b739dbd None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:28.037000 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef51e5-3a82-4975-8516-70841b739dbd None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:28.041506 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b372590-1fe6-4392-ad0f-ffe80f5d59e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:28.042200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b372590-1fe6-4392-ad0f-ffe80f5d59e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:28.042456 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b372590-1fe6-4392-ad0f-ffe80f5d59e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:28.042724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b372590-1fe6-4392-ad0f-ffe80f5d59e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:28.052903 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b372590-1fe6-4392-ad0f-ffe80f5d59e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:28.054340 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b372590-1fe6-4392-ad0f-ffe80f5d59e8 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:28.064342 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 740/1467] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:30.088929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a0d8f-b7a8-4f1b-8224-0906496b0052 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:30.089867 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a0d8f-b7a8-4f1b-8224-0906496b0052 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:30.089867 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a0d8f-b7a8-4f1b-8224-0906496b0052 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:30.092075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 25 16:00:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:30.098353 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fc7bd-26e6-47a5-87de-ca3b13719068 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:30.098794 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fc7bd-26e6-47a5-87de-ca3b13719068 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:30.098921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787fc7bd-26e6-47a5-87de-ca3b13719068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:30.124977 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-787fc7bd-26e6-47a5-87de-ca3b13719068 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:30.126316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-787fc7bd-26e6-47a5-87de-ca3b13719068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:30.135384 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 25 16:00:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:30.176148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46a0d8f-b7a8-4f1b-8224-0906496b0052 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:30.193338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46a0d8f-b7a8-4f1b-8224-0906496b0052 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:30.197594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495b4b9-1a61-4269-baf5-16ac8ae6ef3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:30.198264 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495b4b9-1a61-4269-baf5-16ac8ae6ef3d service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:30.198562 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495b4b9-1a61-4269-baf5-16ac8ae6ef3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:30.198854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495b4b9-1a61-4269-baf5-16ac8ae6ef3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:30.208148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495b4b9-1a61-4269-baf5-16ac8ae6ef3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:30.209605 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495b4b9-1a61-4269-baf5-16ac8ae6ef3d service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:30.222770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 729/1470] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:40.116480 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=11 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:00:40.214787 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=14 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:00:50.027891 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702824f-ca67-493c-b239-36acbc9b08d1 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:50.028511 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702824f-ca67-493c-b239-36acbc9b08d1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:50.028511 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702824f-ca67-493c-b239-36acbc9b08d1 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:50.029861 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 742/1471] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 16:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:50.036695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f74c2-b04a-4973-a3cf-7b65e4cda912 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:50.037175 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f74c2-b04a-4973-a3cf-7b65e4cda912 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:50.037288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f74c2-b04a-4973-a3cf-7b65e4cda912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:50.064869 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-806f74c2-b04a-4973-a3cf-7b65e4cda912 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:50.066031 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-806f74c2-b04a-4973-a3cf-7b65e4cda912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:50.074537 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 730/1472] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 16:00:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:50.128172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b591f-d18f-40d7-b5ac-5f31c25cc399 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:50.128490 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b591f-d18f-40d7-b5ac-5f31c25cc399 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:50.128771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b591f-d18f-40d7-b5ac-5f31c25cc399 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:50.130163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 743/1473] 10.222.0.13 () {54 vars in 962 bytes} [Sat Jun 25 16:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:50.135102 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6e2b1-264b-414a-893e-12740cca99bc None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:50.135388 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6e2b1-264b-414a-893e-12740cca99bc None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:50.135662 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6e2b1-264b-414a-893e-12740cca99bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:50.155570 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e4d6e2b1-264b-414a-893e-12740cca99bc None None] MFA Rules not processed for user `5cba086728824ec68fb0ee90fb3f99dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:50.156883 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e4d6e2b1-264b-414a-893e-12740cca99bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:50.164983 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 731/1474] 10.222.0.13 () {58 vars in 1059 bytes} [Sat Jun 25 16:00:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:50.172544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b591f-d18f-40d7-b5ac-5f31c25cc399 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:50.188985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b591f-d18f-40d7-b5ac-5f31c25cc399 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:50.192877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b3148-30e1-4458-b20b-6baeae6bb2e8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cba086728824ec68fb0ee90fb3f99dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:50.193505 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b3148-30e1-4458-b20b-6baeae6bb2e8 service glance] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:50.193793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b3148-30e1-4458-b20b-6baeae6bb2e8 service glance] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:50.194066 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b3148-30e1-4458-b20b-6baeae6bb2e8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:50.203456 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b3148-30e1-4458-b20b-6baeae6bb2e8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:50.204917 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b3148-30e1-4458-b20b-6baeae6bb2e8 service glance] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:50.214422 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 744/1475] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:52.337108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43d681-fad7-489b-abf9-dc2fec72ece0 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:52.337725 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43d681-fad7-489b-abf9-dc2fec72ece0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:52.337725 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43d681-fad7-489b-abf9-dc2fec72ece0 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:52.339014 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 732/1476] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 16:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:52.344447 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f875dee-3121-4857-ac8e-f2bfc3301723 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:52.344882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f875dee-3121-4857-ac8e-f2bfc3301723 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:52.345008 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f875dee-3121-4857-ac8e-f2bfc3301723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:52.372250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7f875dee-3121-4857-ac8e-f2bfc3301723 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:52.373485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7f875dee-3121-4857-ac8e-f2bfc3301723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:52.383708 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 745/1477] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 16:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:52.415955 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43d681-fad7-489b-abf9-dc2fec72ece0 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:52.438110 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43d681-fad7-489b-abf9-dc2fec72ece0 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:52.442911 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f502ae2-67c2-45be-8a33-8a7c6f461045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:52.443569 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f502ae2-67c2-45be-8a33-8a7c6f461045 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:52.443859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f502ae2-67c2-45be-8a33-8a7c6f461045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:52.444137 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f502ae2-67c2-45be-8a33-8a7c6f461045 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:52.444930 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f502ae2-67c2-45be-8a33-8a7c6f461045 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:52.446842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: WARNING py.warnings [None req-0f502ae2-67c2-45be-8a33-8a7c6f461045 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 25 16:00:52.446842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: warnings.warn(msg) Jun 25 16:00:52.446842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: Jun 25 16:00:52.447996 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f502ae2-67c2-45be-8a33-8a7c6f461045 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:52.467900 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 733/1478] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Jun 25 16:00:52 2022] GET /identity/v3/projects => generated 2553 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:00:54.535814 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915f0bd-c8c6-4356-bcfa-bffee8049c54 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:54.536809 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915f0bd-c8c6-4356-bcfa-bffee8049c54 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:54.536809 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915f0bd-c8c6-4356-bcfa-bffee8049c54 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:54.539021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 746/1479] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 16:00:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:54.545352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd38e80-e336-4222-b101-da492380d165 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:54.546084 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd38e80-e336-4222-b101-da492380d165 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:54.546255 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd38e80-e336-4222-b101-da492380d165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:54.572086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-cdd38e80-e336-4222-b101-da492380d165 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:54.573314 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-cdd38e80-e336-4222-b101-da492380d165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:54.582779 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 734/1480] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 16:00:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:54.625551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915f0bd-c8c6-4356-bcfa-bffee8049c54 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:54.641028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915f0bd-c8c6-4356-bcfa-bffee8049c54 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:54.646612 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa012c-069d-4a7c-9fe8-e282f8ba2013 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:54.647244 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa012c-069d-4a7c-9fe8-e282f8ba2013 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:54.647524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa012c-069d-4a7c-9fe8-e282f8ba2013 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:54.647794 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa012c-069d-4a7c-9fe8-e282f8ba2013 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:54.656592 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa012c-069d-4a7c-9fe8-e282f8ba2013 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:54.658184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa012c-069d-4a7c-9fe8-e282f8ba2013 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:54.667599 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 747/1481] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:00:56.879036 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0dbca-9e96-47b5-8f46-4a8549a829ab None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:56.879710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0dbca-9e96-47b5-8f46-4a8549a829ab None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:56.879807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0dbca-9e96-47b5-8f46-4a8549a829ab None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:56.881979 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 735/1482] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 16:00:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:56.887399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f85c9-c357-400f-9acf-3a0e0bc041fb None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:56.887850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f85c9-c357-400f-9acf-3a0e0bc041fb None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:56.887971 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f85c9-c357-400f-9acf-3a0e0bc041fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:56.909383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f48f85c9-c357-400f-9acf-3a0e0bc041fb None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:56.910587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f48f85c9-c357-400f-9acf-3a0e0bc041fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:56.920071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 748/1483] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 16:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:56.958103 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0dbca-9e96-47b5-8f46-4a8549a829ab None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:56.970638 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0dbca-9e96-47b5-8f46-4a8549a829ab None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:56.974955 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e27f6a-c4a7-4b14-8b95-52ff8f0292d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:56.975588 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e27f6a-c4a7-4b14-8b95-52ff8f0292d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:56.975858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e27f6a-c4a7-4b14-8b95-52ff8f0292d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:56.976122 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e27f6a-c4a7-4b14-8b95-52ff8f0292d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:56.986271 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e27f6a-c4a7-4b14-8b95-52ff8f0292d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:56.987702 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e27f6a-c4a7-4b14-8b95-52ff8f0292d1 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:56.997309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 736/1484] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:00:59.049768 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfa413-1bf2-4ee4-975f-37c0580a5b06 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:59.050626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfa413-1bf2-4ee4-975f-37c0580a5b06 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:59.050626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfa413-1bf2-4ee4-975f-37c0580a5b06 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:59.052955 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 749/1485] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 16:00:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:00:59.060724 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094ab03-b209-48a8-b9e3-f3a7916ec996 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:59.061182 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094ab03-b209-48a8-b9e3-f3a7916ec996 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:59.061291 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094ab03-b209-48a8-b9e3-f3a7916ec996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:59.079412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-7094ab03-b209-48a8-b9e3-f3a7916ec996 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:00:59.080635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7094ab03-b209-48a8-b9e3-f3a7916ec996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:00:59.088996 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 737/1486] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 16:00:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:00:59.133313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfa413-1bf2-4ee4-975f-37c0580a5b06 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:00:59.151825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfa413-1bf2-4ee4-975f-37c0580a5b06 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:00:59.157446 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ad047-1eb2-42e9-a986-b95da578f102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:00:59.158111 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ad047-1eb2-42e9-a986-b95da578f102 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:00:59.158367 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ad047-1eb2-42e9-a986-b95da578f102 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:00:59.158647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ad047-1eb2-42e9-a986-b95da578f102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:00:59.168449 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ad047-1eb2-42e9-a986-b95da578f102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:00:59.169884 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ad047-1eb2-42e9-a986-b95da578f102 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:00:59.180057 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 750/1487] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:00:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:01:01.307915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91bc65-3b87-41d7-8c26-ab3102b4a580 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:01:01.308874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91bc65-3b87-41d7-8c26-ab3102b4a580 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:01:01.308874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91bc65-3b87-41d7-8c26-ab3102b4a580 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:01:01.311066 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 738/1488] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 16:01:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:01:01.319600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b72d7-b3ba-47f0-aed5-661768b731f0 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:01:01.320038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b72d7-b3ba-47f0-aed5-661768b731f0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:01:01.320148 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b72d7-b3ba-47f0-aed5-661768b731f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:01:01.346481 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b89b72d7-b3ba-47f0-aed5-661768b731f0 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:01:01.347673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b89b72d7-b3ba-47f0-aed5-661768b731f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:01:01.356602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 751/1489] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 16:01:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:01:01.400533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91bc65-3b87-41d7-8c26-ab3102b4a580 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:01:01.414715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91bc65-3b87-41d7-8c26-ab3102b4a580 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:01:01.418682 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de5e52-8550-4441-b28d-2dfee8352777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:01:01.419284 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de5e52-8550-4441-b28d-2dfee8352777 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:01:01.419556 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de5e52-8550-4441-b28d-2dfee8352777 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:01:01.419823 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de5e52-8550-4441-b28d-2dfee8352777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:01:01.429349 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de5e52-8550-4441-b28d-2dfee8352777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:01:01.430799 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de5e52-8550-4441-b28d-2dfee8352777 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:01:01.441099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 739/1490] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:01:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:01:03.672770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffbf05-e397-4dac-8d1c-38199b55da4d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:01:03.673400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffbf05-e397-4dac-8d1c-38199b55da4d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:01:03.673400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffbf05-e397-4dac-8d1c-38199b55da4d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:01:03.674686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 752/1491] 10.222.0.13 () {54 vars in 926 bytes} [Sat Jun 25 16:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:01:03.681052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf23c6b-fa05-46cd-be1f-3fba1877b723 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:01:03.681848 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf23c6b-fa05-46cd-be1f-3fba1877b723 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:01:03.682034 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf23c6b-fa05-46cd-be1f-3fba1877b723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:01:03.702998 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-daf23c6b-fa05-46cd-be1f-3fba1877b723 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:01:03.704214 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-daf23c6b-fa05-46cd-be1f-3fba1877b723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:01:03.713281 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 740/1492] 10.222.0.13 () {58 vars in 1023 bytes} [Sat Jun 25 16:01:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:01:03.751551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ffbf05-e397-4dac-8d1c-38199b55da4d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:01:03.764078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ffbf05-e397-4dac-8d1c-38199b55da4d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:01:03.768568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f543f8-59e9-415a-9a9c-668ddba1d62b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:01:03.769210 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f543f8-59e9-415a-9a9c-668ddba1d62b service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:01:03.769489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f543f8-59e9-415a-9a9c-668ddba1d62b service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:01:03.769774 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f543f8-59e9-415a-9a9c-668ddba1d62b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:01:03.778823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f543f8-59e9-415a-9a9c-668ddba1d62b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:01:03.780240 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f543f8-59e9-415a-9a9c-668ddba1d62b service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:01:03.789567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 753/1493] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:01:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:01:13.697747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=33 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:01:13.785582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=33 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:02:28.478176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fdaef-8abf-4ed1-b3aa-a71e5335bba2 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:02:28.478796 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fdaef-8abf-4ed1-b3aa-a71e5335bba2 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:02:28.478796 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fdaef-8abf-4ed1-b3aa-a71e5335bba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:02:28.503033 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-759fdaef-8abf-4ed1-b3aa-a71e5335bba2 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:02:28.504256 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-759fdaef-8abf-4ed1-b3aa-a71e5335bba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:02:28.513784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 741/1494] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:02:28.522227 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e516948-7857-44c1-92c2-eb68d3e122b8 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:02:28.522757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e516948-7857-44c1-92c2-eb68d3e122b8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:02:28.522757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e516948-7857-44c1-92c2-eb68d3e122b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:02:28.542350 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5e516948-7857-44c1-92c2-eb68d3e122b8 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:02:28.543554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5e516948-7857-44c1-92c2-eb68d3e122b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:02:28.552324 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 754/1495] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:02:28.557141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fdaef-8abf-4ed1-b3aa-a71e5335bba2 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:02:28.567517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fdaef-8abf-4ed1-b3aa-a71e5335bba2 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:02:28.573368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d09b8-00cb-470d-8e05-0284f904b545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:02:28.574014 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d09b8-00cb-470d-8e05-0284f904b545 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:02:28.574289 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d09b8-00cb-470d-8e05-0284f904b545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:02:28.574553 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d09b8-00cb-470d-8e05-0284f904b545 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:02:28.575275 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d09b8-00cb-470d-8e05-0284f904b545 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:02:28.575548 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d09b8-00cb-470d-8e05-0284f904b545 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:02:28.576626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: WARNING py.warnings [None req-287d09b8-00cb-470d-8e05-0284f904b545 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 25 16:02:28.576626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: warnings.warn(msg) Jun 25 16:02:28.576626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: Jun 25 16:02:28.577589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d09b8-00cb-470d-8e05-0284f904b545 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:02:28.589833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 742/1496] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:02:28.601030 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e516948-7857-44c1-92c2-eb68d3e122b8 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:02:28.610865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e516948-7857-44c1-92c2-eb68d3e122b8 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:02:28.617955 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e960c-fd7d-4bb9-b894-1e42991d8cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:02:28.618637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e960c-fd7d-4bb9-b894-1e42991d8cbd admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:02:28.618916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e960c-fd7d-4bb9-b894-1e42991d8cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:02:28.619198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e960c-fd7d-4bb9-b894-1e42991d8cbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:02:28.620193 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e960c-fd7d-4bb9-b894-1e42991d8cbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:02:28.623166 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: WARNING py.warnings [None req-c96e960c-fd7d-4bb9-b894-1e42991d8cbd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 25 16:02:28.623166 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: warnings.warn(msg) Jun 25 16:02:28.623166 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: Jun 25 16:02:28.624368 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e960c-fd7d-4bb9-b894-1e42991d8cbd admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:02:28.661956 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 755/1497] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:02:28 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:02:28.666530 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d09b8-00cb-470d-8e05-0284f904b545 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:02:28.676909 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d09b8-00cb-470d-8e05-0284f904b545 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:02:28.736193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ea52e-253b-4032-b37e-8640037c798d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:02:28.736841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ea52e-253b-4032-b37e-8640037c798d admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:02:28.737131 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ea52e-253b-4032-b37e-8640037c798d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:02:28.737382 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ea52e-253b-4032-b37e-8640037c798d admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:02:28.738069 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ea52e-253b-4032-b37e-8640037c798d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:02:28.739124 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: WARNING py.warnings [None req-ae2ea52e-253b-4032-b37e-8640037c798d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 25 16:02:28.739124 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: warnings.warn(msg) Jun 25 16:02:28.739124 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: Jun 25 16:02:28.740321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ea52e-253b-4032-b37e-8640037c798d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:02:28.762326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 743/1498] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:02:28 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:02:28.768217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e960c-fd7d-4bb9-b894-1e42991d8cbd admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:02:28.782520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e960c-fd7d-4bb9-b894-1e42991d8cbd admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:02:28.787619 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcaf6f7-b72c-41e3-ae64-09d6edd1148f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:02:28.788268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcaf6f7-b72c-41e3-ae64-09d6edd1148f admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:02:28.788589 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcaf6f7-b72c-41e3-ae64-09d6edd1148f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:02:28.788864 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcaf6f7-b72c-41e3-ae64-09d6edd1148f admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:02:28.789578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcaf6f7-b72c-41e3-ae64-09d6edd1148f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:02:28.790581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: WARNING py.warnings [None req-abcaf6f7-b72c-41e3-ae64-09d6edd1148f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 25 16:02:28.790581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: warnings.warn(msg) Jun 25 16:02:28.790581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: Jun 25 16:02:28.791451 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcaf6f7-b72c-41e3-ae64-09d6edd1148f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:02:28.804352 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 756/1499] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:02:28.808395 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ea52e-253b-4032-b37e-8640037c798d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:02:28.818509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ea52e-253b-4032-b37e-8640037c798d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:02:28.823585 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebeabf1-09de-4f7e-b34b-766b89c2fdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:02:28.824230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebeabf1-09de-4f7e-b34b-766b89c2fdb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:02:28.824540 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebeabf1-09de-4f7e-b34b-766b89c2fdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:02:28.824811 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebeabf1-09de-4f7e-b34b-766b89c2fdb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:02:28.825576 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebeabf1-09de-4f7e-b34b-766b89c2fdb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:02:28.826599 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: WARNING py.warnings [None req-2ebeabf1-09de-4f7e-b34b-766b89c2fdb0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 25 16:02:28.826599 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: warnings.warn(msg) Jun 25 16:02:28.826599 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: Jun 25 16:02:28.827456 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebeabf1-09de-4f7e-b34b-766b89c2fdb0 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:02:28.841078 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 744/1500] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:02:28.845500 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcaf6f7-b72c-41e3-ae64-09d6edd1148f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:02:28.857964 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcaf6f7-b72c-41e3-ae64-09d6edd1148f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:02:28.863250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41438b04-0017-488b-bd56-91ae7e09ef7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:02:28.864147 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41438b04-0017-488b-bd56-91ae7e09ef7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:02:28.864449 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41438b04-0017-488b-bd56-91ae7e09ef7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:02:28.864731 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41438b04-0017-488b-bd56-91ae7e09ef7f admin admin] PATH_INFO: `/v3/projects/866ec87d31364266ae396af164eb83de/users/d664029ee2e949a6abb0ffcaa939e06d/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:02:28.880460 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41438b04-0017-488b-bd56-91ae7e09ef7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=866ec87d31364266ae396af164eb83de, user_id=d664029ee2e949a6abb0ffcaa939e06d, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:02:28.881503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: WARNING py.warnings [None req-41438b04-0017-488b-bd56-91ae7e09ef7f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 25 16:02:28.881503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: warnings.warn(msg) Jun 25 16:02:28.881503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: Jun 25 16:02:28.883424 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41438b04-0017-488b-bd56-91ae7e09ef7f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:02:28.898531 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 757/1501] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:02:28 2022] PUT /identity/v3/projects/866ec87d31364266ae396af164eb83de/users/d664029ee2e949a6abb0ffcaa939e06d/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:02:28.910102 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39458f-6ded-4d3b-9bd5-5ae68113a334 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:02:28.910386 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39458f-6ded-4d3b-9bd5-5ae68113a334 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:02:28.910657 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39458f-6ded-4d3b-9bd5-5ae68113a334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:02:28.944720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ce39458f-6ded-4d3b-9bd5-5ae68113a334 None None] MFA Rules not processed for user `d664029ee2e949a6abb0ffcaa939e06d`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:02:28.946087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ce39458f-6ded-4d3b-9bd5-5ae68113a334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:02:29.013353 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 745/1502] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:02:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:02:29.027944 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4cea0-bb8e-40da-a8f2-3a16a06e3528 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:02:29.028345 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4cea0-bb8e-40da-a8f2-3a16a06e3528 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:02:29.028661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4cea0-bb8e-40da-a8f2-3a16a06e3528 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:02:29.030194 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 758/1503] 10.222.0.13 () {54 vars in 963 bytes} [Sat Jun 25 16:02:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:02:29.035091 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d2069-26f6-4f7c-98df-751b36cd1f27 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:02:29.035378 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d2069-26f6-4f7c-98df-751b36cd1f27 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:02:29.035654 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d2069-26f6-4f7c-98df-751b36cd1f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:02:29.054890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-357d2069-26f6-4f7c-98df-751b36cd1f27 None None] MFA Rules not processed for user `b00152c72a1f4755b09c1dbd1d874b11`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:02:29.056091 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-357d2069-26f6-4f7c-98df-751b36cd1f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:02:29.101193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:02:29.110350 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 746/1504] 10.222.0.13 () {58 vars in 1060 bytes} [Sat Jun 25 16:02:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:02:29.118762 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4cea0-bb8e-40da-a8f2-3a16a06e3528 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:02:29.135317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4cea0-bb8e-40da-a8f2-3a16a06e3528 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:02:29.139967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bea892-45d0-428b-bc2b-53495b479c67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:02:29.140951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bea892-45d0-428b-bc2b-53495b479c67 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:02:29.141233 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bea892-45d0-428b-bc2b-53495b479c67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:02:29.141510 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bea892-45d0-428b-bc2b-53495b479c67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:02:29.150083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bea892-45d0-428b-bc2b-53495b479c67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:02:29.151502 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bea892-45d0-428b-bc2b-53495b479c67 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:02:29.160708 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 759/1505] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:02:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:02:29.202204 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5f5d4-dad4-4529-a936-a328557768e5 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:02:29.202491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5f5d4-dad4-4529-a936-a328557768e5 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:02:29.202759 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5f5d4-dad4-4529-a936-a328557768e5 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:02:29.204044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 747/1506] 10.222.0.13 () {52 vars in 829 bytes} [Sat Jun 25 16:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:02:29.207735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bea892-45d0-428b-bc2b-53495b479c67 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:02:29.271054 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bea892-45d0-428b-bc2b-53495b479c67 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:02:29.275941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30506c6f-4d42-427f-bb26-fd3be33f99c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:02:29.276836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30506c6f-4d42-427f-bb26-fd3be33f99c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:02:29.277081 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30506c6f-4d42-427f-bb26-fd3be33f99c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:02:29.277386 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30506c6f-4d42-427f-bb26-fd3be33f99c4 admin admin] PATH_INFO: `/v3/users/d664029ee2e949a6abb0ffcaa939e06d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:02:29.278645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30506c6f-4d42-427f-bb26-fd3be33f99c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d664029ee2e949a6abb0ffcaa939e06d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:02:29.279842 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: WARNING py.warnings [None req-30506c6f-4d42-427f-bb26-fd3be33f99c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 25 16:02:29.279842 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: warnings.warn(msg) Jun 25 16:02:29.279842 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: Jun 25 16:02:29.281107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30506c6f-4d42-427f-bb26-fd3be33f99c4 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:02:29.351499 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-30506c6f-4d42-427f-bb26-fd3be33f99c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd664029ee2e949a6abb0ffcaa939e06d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:02:29.364589 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-30506c6f-4d42-427f-bb26-fd3be33f99c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd664029ee2e949a6abb0ffcaa939e06d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:02:29.365422 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-30506c6f-4d42-427f-bb26-fd3be33f99c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd664029ee2e949a6abb0ffcaa939e06d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:02:29.381274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-30506c6f-4d42-427f-bb26-fd3be33f99c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd664029ee2e949a6abb0ffcaa939e06d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:02:29.382259 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-30506c6f-4d42-427f-bb26-fd3be33f99c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd664029ee2e949a6abb0ffcaa939e06d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:02:29.407102 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:02:29.416206 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 760/1507] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:02:29 2022] DELETE /identity/v3/users/d664029ee2e949a6abb0ffcaa939e06d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:02:29.420750 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121699b-e8f5-4979-97c1-da8e1afbc7a3 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:02:29.421037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121699b-e8f5-4979-97c1-da8e1afbc7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:02:29.421316 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121699b-e8f5-4979-97c1-da8e1afbc7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:02:29.439290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-4121699b-e8f5-4979-97c1-da8e1afbc7a3 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:02:29.440557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4121699b-e8f5-4979-97c1-da8e1afbc7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:02:29.510834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 748/1508] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:02:29.522081 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30506c6f-4d42-427f-bb26-fd3be33f99c4 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:02:29.523604 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-30506c6f-4d42-427f-bb26-fd3be33f99c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:02:29.584527 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30506c6f-4d42-427f-bb26-fd3be33f99c4 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:02:29.588604 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b117cb-aa91-4f8b-b639-a6678aafdce8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:02:29.589246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b117cb-aa91-4f8b-b639-a6678aafdce8 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:02:29.589534 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b117cb-aa91-4f8b-b639-a6678aafdce8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:02:29.589810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b117cb-aa91-4f8b-b639-a6678aafdce8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:02:29.599561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b117cb-aa91-4f8b-b639-a6678aafdce8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:02:29.601059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b117cb-aa91-4f8b-b639-a6678aafdce8 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:02:29.611351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 761/1509] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:02:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:02:29.999990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121699b-e8f5-4979-97c1-da8e1afbc7a3 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:02:30.001675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4121699b-e8f5-4979-97c1-da8e1afbc7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:02:30.014028 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121699b-e8f5-4979-97c1-da8e1afbc7a3 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:02:30.019811 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9e84b-f188-4966-956e-5b7ebda67275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:02:30.020744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9e84b-f188-4966-956e-5b7ebda67275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:02:30.021018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9e84b-f188-4966-956e-5b7ebda67275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:02:30.021286 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9e84b-f188-4966-956e-5b7ebda67275 admin admin] PATH_INFO: `/v3/projects/866ec87d31364266ae396af164eb83de` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:02:30.022580 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9e84b-f188-4966-956e-5b7ebda67275 admin admin] RBAC: Authorizing `identity:delete_project(project_id=866ec87d31364266ae396af164eb83de)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:02:30.023610 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: WARNING py.warnings [None req-0ac9e84b-f188-4966-956e-5b7ebda67275 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 25 16:02:30.023610 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: warnings.warn(msg) Jun 25 16:02:30.023610 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: Jun 25 16:02:30.024850 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9e84b-f188-4966-956e-5b7ebda67275 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:02:30.090027 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0ac9e84b-f188-4966-956e-5b7ebda67275 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '866ec87d31364266ae396af164eb83de', 'request_id': 'req-0ac9e84b-f188-4966-956e-5b7ebda67275'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:02:30.096931 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0ac9e84b-f188-4966-956e-5b7ebda67275 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '866ec87d31364266ae396af164eb83de', 'request_id': 'req-0ac9e84b-f188-4966-956e-5b7ebda67275'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:02:30.111040 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0ac9e84b-f188-4966-956e-5b7ebda67275 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '866ec87d31364266ae396af164eb83de', 'request_id': 'req-0ac9e84b-f188-4966-956e-5b7ebda67275'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:02:30.129981 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0ac9e84b-f188-4966-956e-5b7ebda67275 admin admin] The token cache is being invalidate because project 866ec87d31364266ae396af164eb83de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:02:30.130293 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0ac9e84b-f188-4966-956e-5b7ebda67275 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:02:30.131411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0ac9e84b-f188-4966-956e-5b7ebda67275 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:02:30.134217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 749/1510] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:02:29 2022] DELETE /identity/v3/projects/866ec87d31364266ae396af164eb83de => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:02:39.606431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:02:40.125933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:04:04.993488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411e272-45d6-45c5-b6f2-8cafaa16ff01 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:04.994131 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411e272-45d6-45c5-b6f2-8cafaa16ff01 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:04.994131 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411e272-45d6-45c5-b6f2-8cafaa16ff01 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:04.995611 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 25 16:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:04:05.003116 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160c161-b41b-446f-8399-e8ffb8c48ce1 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:05.003918 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160c161-b41b-446f-8399-e8ffb8c48ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:05.004108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160c161-b41b-446f-8399-e8ffb8c48ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:05.024452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-8160c161-b41b-446f-8399-e8ffb8c48ce1 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:04:05.025667 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8160c161-b41b-446f-8399-e8ffb8c48ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:04:05.086172 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 25 16:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:04:05.240438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411e272-45d6-45c5-b6f2-8cafaa16ff01 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:04:05.243178 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9411e272-45d6-45c5-b6f2-8cafaa16ff01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:04:05.324443 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411e272-45d6-45c5-b6f2-8cafaa16ff01 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:04:05.330664 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774a896-3edc-4837-aa59-6c9e3f06b8de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:04:05.331324 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774a896-3edc-4837-aa59-6c9e3f06b8de service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:05.331600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774a896-3edc-4837-aa59-6c9e3f06b8de service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:05.331870 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774a896-3edc-4837-aa59-6c9e3f06b8de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:05.340941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774a896-3edc-4837-aa59-6c9e3f06b8de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:04:05.342523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774a896-3edc-4837-aa59-6c9e3f06b8de service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:04:05.352068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 763/1513] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:04:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:04:06.174287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc2753-b02b-46f0-8ba9-e66665017823 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:06.175126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc2753-b02b-46f0-8ba9-e66665017823 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:06.175126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc2753-b02b-46f0-8ba9-e66665017823 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:06.177270 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 25 16:04:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:04:06.185074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a4c73-db07-45c0-b0a0-b3f8414c9e0f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:06.185499 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a4c73-db07-45c0-b0a0-b3f8414c9e0f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:06.185612 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a4c73-db07-45c0-b0a0-b3f8414c9e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:06.205104 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3f0a4c73-db07-45c0-b0a0-b3f8414c9e0f None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:04:06.206298 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3f0a4c73-db07-45c0-b0a0-b3f8414c9e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:04:06.214731 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 25 16:04:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:04:06.385056 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc2753-b02b-46f0-8ba9-e66665017823 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:04:06.401865 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc2753-b02b-46f0-8ba9-e66665017823 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:04:06.407509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef674f9-4ddc-4cdd-8d7a-8a16eb7a4e4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:04:06.408163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef674f9-4ddc-4cdd-8d7a-8a16eb7a4e4d service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:06.408442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef674f9-4ddc-4cdd-8d7a-8a16eb7a4e4d service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:06.408738 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef674f9-4ddc-4cdd-8d7a-8a16eb7a4e4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:06.417688 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef674f9-4ddc-4cdd-8d7a-8a16eb7a4e4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:04:06.419281 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef674f9-4ddc-4cdd-8d7a-8a16eb7a4e4d service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:04:06.428819 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 752/1516] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:04:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:04:07.141453 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6dbb4-c757-4214-be5c-00db5f325f19 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:07.141679 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6dbb4-c757-4214-be5c-00db5f325f19 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:07.141907 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6dbb4-c757-4214-be5c-00db5f325f19 None None] PATH_INFO: `/v3` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:07.143275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 765/1517] 10.222.0.26 () {54 vars in 925 bytes} [Sat Jun 25 16:04:07 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:04:07.149925 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdf5a3-f039-44da-82e9-5ddb26d6019b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:07.150209 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdf5a3-f039-44da-82e9-5ddb26d6019b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:07.150485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdf5a3-f039-44da-82e9-5ddb26d6019b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:07.169567 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-1fbdf5a3-f039-44da-82e9-5ddb26d6019b None None] MFA Rules not processed for user `474c468a610140f8a5b18d5a1ed1d18e`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:04:07.170771 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1fbdf5a3-f039-44da-82e9-5ddb26d6019b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:04:07.239671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 753/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Jun 25 16:04:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:04:11.073563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce9d1b-4d34-49a6-8891-850b224b151b None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:11.074232 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce9d1b-4d34-49a6-8891-850b224b151b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:11.074232 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce9d1b-4d34-49a6-8891-850b224b151b None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:11.075415 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 766/1519] 10.222.0.13 () {54 vars in 960 bytes} [Sat Jun 25 16:04:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:04:11.080515 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d57a9-97c8-41bb-a361-d24fc482774a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:11.080999 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d57a9-97c8-41bb-a361-d24fc482774a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:11.081099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d57a9-97c8-41bb-a361-d24fc482774a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:11.108209 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-738d57a9-97c8-41bb-a361-d24fc482774a None None] MFA Rules not processed for user `474c468a610140f8a5b18d5a1ed1d18e`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:04:11.109478 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-738d57a9-97c8-41bb-a361-d24fc482774a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:04:11.120310 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 754/1520] 10.222.0.13 () {58 vars in 1057 bytes} [Sat Jun 25 16:04:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:04:11.129714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce9d1b-4d34-49a6-8891-850b224b151b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:04:11.146718 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce9d1b-4d34-49a6-8891-850b224b151b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:04:11.152049 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3af55-2c26-4359-9b53-46fde04fffc5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474c468a610140f8a5b18d5a1ed1d18e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:04:11.152720 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3af55-2c26-4359-9b53-46fde04fffc5 service placement] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:11.152998 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3af55-2c26-4359-9b53-46fde04fffc5 service placement] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:11.153270 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3af55-2c26-4359-9b53-46fde04fffc5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:11.161997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3af55-2c26-4359-9b53-46fde04fffc5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:04:11.163568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3af55-2c26-4359-9b53-46fde04fffc5 service placement] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:04:11.173568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 767/1521] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:04:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:04:17.694326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58f6c8-ff26-4207-9e5f-2204ed748ebd None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:17.694996 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58f6c8-ff26-4207-9e5f-2204ed748ebd None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:17.694996 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58f6c8-ff26-4207-9e5f-2204ed748ebd None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:17.696470 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 25 16:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:04:17.702278 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f4a92-fed9-47db-a5c0-e3d69c3e4529 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:17.702720 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f4a92-fed9-47db-a5c0-e3d69c3e4529 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:17.702829 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f4a92-fed9-47db-a5c0-e3d69c3e4529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:17.729922 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-723f4a92-fed9-47db-a5c0-e3d69c3e4529 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:04:17.731104 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-723f4a92-fed9-47db-a5c0-e3d69c3e4529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:04:17.739933 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 25 16:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:04:17.895150 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58f6c8-ff26-4207-9e5f-2204ed748ebd None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:04:17.916473 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58f6c8-ff26-4207-9e5f-2204ed748ebd None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:04:17.922860 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb959912-5b16-4985-8c27-d9382a031841 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:04:17.923507 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb959912-5b16-4985-8c27-d9382a031841 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:17.923787 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb959912-5b16-4985-8c27-d9382a031841 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:17.924062 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb959912-5b16-4985-8c27-d9382a031841 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:17.933025 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb959912-5b16-4985-8c27-d9382a031841 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:04:17.934661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb959912-5b16-4985-8c27-d9382a031841 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:04:17.944582 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 756/1524] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:04:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:04:18.762967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392d126-e7fe-4510-a9bc-f1541a2218e8 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:18.763571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392d126-e7fe-4510-a9bc-f1541a2218e8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:18.763571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392d126-e7fe-4510-a9bc-f1541a2218e8 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:18.765015 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jun 25 16:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:04:18.770867 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead54d4-d796-4a98-a181-7cb1e376f4eb None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:18.771335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead54d4-d796-4a98-a181-7cb1e376f4eb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:18.771443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead54d4-d796-4a98-a181-7cb1e376f4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:18.791009 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-7ead54d4-d796-4a98-a181-7cb1e376f4eb None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:04:18.792219 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7ead54d4-d796-4a98-a181-7cb1e376f4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:04:18.801286 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jun 25 16:04:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:04:18.952910 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392d126-e7fe-4510-a9bc-f1541a2218e8 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:04:18.976241 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392d126-e7fe-4510-a9bc-f1541a2218e8 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:04:18.982771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0243e18-7a2a-4260-bb67-62c52a2a3f33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:04:18.983406 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0243e18-7a2a-4260-bb67-62c52a2a3f33 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:18.983684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0243e18-7a2a-4260-bb67-62c52a2a3f33 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:18.983961 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0243e18-7a2a-4260-bb67-62c52a2a3f33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:18.992989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0243e18-7a2a-4260-bb67-62c52a2a3f33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:04:18.994561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0243e18-7a2a-4260-bb67-62c52a2a3f33 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:04:19.004635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 770/1527] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:04:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:04:22.889886 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836f23c-d424-415b-bb77-093d4e1343d7 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:22.890792 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836f23c-d424-415b-bb77-093d4e1343d7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:22.890792 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836f23c-d424-415b-bb77-093d4e1343d7 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:22.892135 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 25 16:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:04:22.897845 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ee1f7-c90e-44ba-8b06-0d8aef3d7f05 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:22.898278 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ee1f7-c90e-44ba-8b06-0d8aef3d7f05 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:22.898578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ee1f7-c90e-44ba-8b06-0d8aef3d7f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:22.919138 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8b4ee1f7-c90e-44ba-8b06-0d8aef3d7f05 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:04:22.920371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8b4ee1f7-c90e-44ba-8b06-0d8aef3d7f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:04:22.929131 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 25 16:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:04:22.968115 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836f23c-d424-415b-bb77-093d4e1343d7 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:04:22.970454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-0836f23c-d424-415b-bb77-093d4e1343d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:04:23.030943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836f23c-d424-415b-bb77-093d4e1343d7 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:04:23.034846 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be302a-28dd-424e-a804-d09d271fb976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:04:23.035494 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be302a-28dd-424e-a804-d09d271fb976 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:23.035776 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be302a-28dd-424e-a804-d09d271fb976 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:23.036048 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be302a-28dd-424e-a804-d09d271fb976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:23.045651 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be302a-28dd-424e-a804-d09d271fb976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:04:23.047122 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be302a-28dd-424e-a804-d09d271fb976 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:04:23.052206 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:04:23.057315 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 759/1530] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:04:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:04:25.055036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e509e9-1867-4916-a036-7917350ecf66 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:25.055920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e509e9-1867-4916-a036-7917350ecf66 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:25.055920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e509e9-1867-4916-a036-7917350ecf66 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:25.057367 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 25 16:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:04:25.063767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b76f2-5dcf-40bb-b903-765d9eb2b27d None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:25.064517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b76f2-5dcf-40bb-b903-765d9eb2b27d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:25.064678 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b76f2-5dcf-40bb-b903-765d9eb2b27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:25.091572 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a23b76f2-5dcf-40bb-b903-765d9eb2b27d None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:04:25.092826 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a23b76f2-5dcf-40bb-b903-765d9eb2b27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:04:25.102211 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 25 16:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:04:25.138627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e509e9-1867-4916-a036-7917350ecf66 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:04:25.156190 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e509e9-1867-4916-a036-7917350ecf66 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:04:25.160927 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f8775e-571e-4bcb-bf35-db9b43283fcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:04:25.161564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8775e-571e-4bcb-bf35-db9b43283fcc service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:25.161847 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8775e-571e-4bcb-bf35-db9b43283fcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:25.162150 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8775e-571e-4bcb-bf35-db9b43283fcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:25.171718 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f8775e-571e-4bcb-bf35-db9b43283fcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:04:25.173170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f8775e-571e-4bcb-bf35-db9b43283fcc service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:04:25.182735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 773/1533] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:04:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:04:28.624447 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecfaaac-4639-4ea9-890b-4a29f312dfa7 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:28.625126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecfaaac-4639-4ea9-890b-4a29f312dfa7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:28.625126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecfaaac-4639-4ea9-890b-4a29f312dfa7 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:28.626364 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 25 16:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 16:04:28.633269 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ef8c8-ead4-45ff-99e5-329794715525 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:28.633916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ef8c8-ead4-45ff-99e5-329794715525 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:28.633916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ef8c8-ead4-45ff-99e5-329794715525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:28.656385 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-bd1ef8c8-ead4-45ff-99e5-329794715525 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:04:28.657600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-bd1ef8c8-ead4-45ff-99e5-329794715525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:04:28.666530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 25 16:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:04:28.977166 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecfaaac-4639-4ea9-890b-4a29f312dfa7 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:04:28.979807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1ecfaaac-4639-4ea9-890b-4a29f312dfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:04:29.041966 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecfaaac-4639-4ea9-890b-4a29f312dfa7 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:04:29.045888 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa027c-a6e8-4e45-b556-781ea9dd6061 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:04:29.046537 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa027c-a6e8-4e45-b556-781ea9dd6061 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:29.046817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa027c-a6e8-4e45-b556-781ea9dd6061 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:29.047087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa027c-a6e8-4e45-b556-781ea9dd6061 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:29.056651 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa027c-a6e8-4e45-b556-781ea9dd6061 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:04:29.058115 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa027c-a6e8-4e45-b556-781ea9dd6061 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:04:29.067512 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 762/1536] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:04:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:04:31.113545 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e742acd-f440-45dc-af38-5a2395e34443 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:31.114397 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e742acd-f440-45dc-af38-5a2395e34443 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:31.114397 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e742acd-f440-45dc-af38-5a2395e34443 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:31.116581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jun 25 16:04:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:04:31.122064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd50cda-c4d1-4a65-ba51-854de1e3a077 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:31.122493 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd50cda-c4d1-4a65-ba51-854de1e3a077 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:31.122609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd50cda-c4d1-4a65-ba51-854de1e3a077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:31.146396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-1bd50cda-c4d1-4a65-ba51-854de1e3a077 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:04:31.147615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1bd50cda-c4d1-4a65-ba51-854de1e3a077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:04:31.158071 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jun 25 16:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:04:31.460089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e742acd-f440-45dc-af38-5a2395e34443 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:04:31.463148 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8e742acd-f440-45dc-af38-5a2395e34443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:04:31.472015 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=65 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:04:31.478606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e742acd-f440-45dc-af38-5a2395e34443 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:04:31.482489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff772d-cd5e-4892-8482-4f9cad56a93e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:04:31.483127 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff772d-cd5e-4892-8482-4f9cad56a93e service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:04:31.483407 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff772d-cd5e-4892-8482-4f9cad56a93e service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:04:31.483681 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff772d-cd5e-4892-8482-4f9cad56a93e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:04:31.492855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff772d-cd5e-4892-8482-4f9cad56a93e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:04:31.494280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff772d-cd5e-4892-8482-4f9cad56a93e service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:04:31.503864 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 776/1539] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:04:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:04:41.138732 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=32 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:04:41.499985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:05:57.486006 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e839df-8c06-4d60-83ab-568c35c5506c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:57.487177 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e839df-8c06-4d60-83ab-568c35c5506c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:57.487680 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e839df-8c06-4d60-83ab-568c35c5506c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:57.512372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a3e839df-8c06-4d60-83ab-568c35c5506c None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:05:57.514177 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a3e839df-8c06-4d60-83ab-568c35c5506c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:05:57.527538 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 764/1540] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:05:57.536807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453e580-a9a4-4052-88ab-5efde5f73d21 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:57.537397 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453e580-a9a4-4052-88ab-5efde5f73d21 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:57.537397 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453e580-a9a4-4052-88ab-5efde5f73d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:57.569483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-2453e580-a9a4-4052-88ab-5efde5f73d21 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:05:57.570948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2453e580-a9a4-4052-88ab-5efde5f73d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:05:57.579946 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 777/1541] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:05:57.584650 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e839df-8c06-4d60-83ab-568c35c5506c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:57.599837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e839df-8c06-4d60-83ab-568c35c5506c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:57.614327 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7c9c0-76bf-40e5-894b-67574e9590f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:57.615382 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7c9c0-76bf-40e5-894b-67574e9590f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:57.615877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7c9c0-76bf-40e5-894b-67574e9590f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:57.616373 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7c9c0-76bf-40e5-894b-67574e9590f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:57.617893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7c9c0-76bf-40e5-894b-67574e9590f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:05:57.618439 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7c9c0-76bf-40e5-894b-67574e9590f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:57.620294 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7c9c0-76bf-40e5-894b-67574e9590f7 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:57.622348 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3c956-09d0-4d71-a735-ee9a1fc95cdd None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:57.622348 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3c956-09d0-4d71-a735-ee9a1fc95cdd None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:57.622348 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3c956-09d0-4d71-a735-ee9a1fc95cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:57.643416 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 765/1542] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:05:57.650109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7c9c0-76bf-40e5-894b-67574e9590f7 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:57.657534 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e7b3c956-09d0-4d71-a735-ee9a1fc95cdd None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:05:57.658769 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e7b3c956-09d0-4d71-a735-ee9a1fc95cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:05:57.663216 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7c9c0-76bf-40e5-894b-67574e9590f7 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:57.671812 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 778/1543] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:05:57.675437 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be2374-a229-4463-bce2-4b31323a76e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:57.676377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be2374-a229-4463-bce2-4b31323a76e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:57.676904 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be2374-a229-4463-bce2-4b31323a76e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:57.677404 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be2374-a229-4463-bce2-4b31323a76e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:57.678518 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be2374-a229-4463-bce2-4b31323a76e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:57.679765 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: WARNING py.warnings [None req-90be2374-a229-4463-bce2-4b31323a76e1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 25 16:05:57.679765 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: warnings.warn(msg) Jun 25 16:05:57.679765 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: Jun 25 16:05:57.680628 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9772da3-48ee-4ba5-b692-0dd5bb3f652f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:57.680918 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9772da3-48ee-4ba5-b692-0dd5bb3f652f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:57.681198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9772da3-48ee-4ba5-b692-0dd5bb3f652f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:57.682098 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be2374-a229-4463-bce2-4b31323a76e1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:57.712467 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e9772da3-48ee-4ba5-b692-0dd5bb3f652f None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:05:57.712831 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e9772da3-48ee-4ba5-b692-0dd5bb3f652f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:05:57.732784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 766/1544] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:05:57 2022] POST /identity/v3/projects => generated 393 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:05:57.737396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be2374-a229-4463-bce2-4b31323a76e1 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:57.741590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 779/1545] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:05:57.749945 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9772da3-48ee-4ba5-b692-0dd5bb3f652f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:57.763852 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9772da3-48ee-4ba5-b692-0dd5bb3f652f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:57.864375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be2374-a229-4463-bce2-4b31323a76e1 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:57.887713 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a01ffa-c3d7-4f46-b2ba-fd8fac974afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:57.889835 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a01ffa-c3d7-4f46-b2ba-fd8fac974afa admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:57.891310 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a01ffa-c3d7-4f46-b2ba-fd8fac974afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:57.893775 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a01ffa-c3d7-4f46-b2ba-fd8fac974afa admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:57.894714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a01ffa-c3d7-4f46-b2ba-fd8fac974afa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:57.896934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a01ffa-c3d7-4f46-b2ba-fd8fac974afa admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:57.909328 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af7fd1-d504-429f-b3bd-39e8ba9f58a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:57.910034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af7fd1-d504-429f-b3bd-39e8ba9f58a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:57.910318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af7fd1-d504-429f-b3bd-39e8ba9f58a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:57.910602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af7fd1-d504-429f-b3bd-39e8ba9f58a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:57.911351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af7fd1-d504-429f-b3bd-39e8ba9f58a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:05:57.911658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af7fd1-d504-429f-b3bd-39e8ba9f58a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:57.912833 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: WARNING py.warnings [None req-88af7fd1-d504-429f-b3bd-39e8ba9f58a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 25 16:05:57.912833 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: warnings.warn(msg) Jun 25 16:05:57.912833 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: Jun 25 16:05:57.913607 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af7fd1-d504-429f-b3bd-39e8ba9f58a6 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:57.947060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 780/1546] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:05:57.951062 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512402f-fa56-4610-9303-6148b76b5735 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:57.951372 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512402f-fa56-4610-9303-6148b76b5735 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:57.951660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512402f-fa56-4610-9303-6148b76b5735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:57.961424 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 767/1547] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:05:57 2022] POST /identity/v3/users => generated 297 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:05:57.962985 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a01ffa-c3d7-4f46-b2ba-fd8fac974afa admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:58.000550 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-2512402f-fa56-4610-9303-6148b76b5735 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:05:58.000550 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2512402f-fa56-4610-9303-6148b76b5735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:05:58.002763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a01ffa-c3d7-4f46-b2ba-fd8fac974afa admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:58.009212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 781/1548] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:05:58.014171 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64417e2-f103-47c2-8265-4ff3a41c4692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:58.015703 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64417e2-f103-47c2-8265-4ff3a41c4692 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.020489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512402f-fa56-4610-9303-6148b76b5735 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:58.022617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64417e2-f103-47c2-8265-4ff3a41c4692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.022919 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64417e2-f103-47c2-8265-4ff3a41c4692 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.023678 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64417e2-f103-47c2-8265-4ff3a41c4692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:58.028567 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64417e2-f103-47c2-8265-4ff3a41c4692 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:58.057878 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512402f-fa56-4610-9303-6148b76b5735 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:58.085396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f11f9d-0dd7-4f45-84ac-14cf188a4dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:58.086856 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f11f9d-0dd7-4f45-84ac-14cf188a4dda admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.086856 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f11f9d-0dd7-4f45-84ac-14cf188a4dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.087064 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f11f9d-0dd7-4f45-84ac-14cf188a4dda admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.087654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f11f9d-0dd7-4f45-84ac-14cf188a4dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:58.089323 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f11f9d-0dd7-4f45-84ac-14cf188a4dda admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:58.118487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 782/1549] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:05:58.120249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa518080-402b-4970-ae38-fe2df5567885 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.123751 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 768/1550] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:05:57 2022] POST /identity/v3/projects => generated 403 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:05:58.124639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64417e2-f103-47c2-8265-4ff3a41c4692 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:58.124796 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa518080-402b-4970-ae38-fe2df5567885 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.125056 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa518080-402b-4970-ae38-fe2df5567885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.160567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-aa518080-402b-4970-ae38-fe2df5567885 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:05:58.161787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-aa518080-402b-4970-ae38-fe2df5567885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:05:58.257685 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64417e2-f103-47c2-8265-4ff3a41c4692 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:58.277432 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d8fb3-976f-4b43-9bcd-f84567d9a1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:58.281194 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d8fb3-976f-4b43-9bcd-f84567d9a1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.281483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d8fb3-976f-4b43-9bcd-f84567d9a1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.281784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d8fb3-976f-4b43-9bcd-f84567d9a1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.282829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d8fb3-976f-4b43-9bcd-f84567d9a1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:58.284708 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d8fb3-976f-4b43-9bcd-f84567d9a1e8 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:58.309498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 769/1551] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:05:58.309498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d8fb3-976f-4b43-9bcd-f84567d9a1e8 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:58.325700 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d8fb3-976f-4b43-9bcd-f84567d9a1e8 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:58.334010 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9118ceac-80b0-4b13-91f6-fc0aad719ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:58.337123 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118ceac-80b0-4b13-91f6-fc0aad719ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.337123 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118ceac-80b0-4b13-91f6-fc0aad719ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.337123 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118ceac-80b0-4b13-91f6-fc0aad719ec7 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.337123 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9118ceac-80b0-4b13-91f6-fc0aad719ec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:58.339071 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9118ceac-80b0-4b13-91f6-fc0aad719ec7 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:58.344750 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 783/1552] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:05:58.346157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e58b6-05bd-4aae-8ac7-ca8a1335a5f0 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.346544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e58b6-05bd-4aae-8ac7-ca8a1335a5f0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.346920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e58b6-05bd-4aae-8ac7-ca8a1335a5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.364822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 770/1553] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:05:58 2022] POST /identity/v3/users => generated 302 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:05:58.366408 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa3d2f-8dd1-4a4a-a6c2-42e8dbc2b940 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.366699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa3d2f-8dd1-4a4a-a6c2-42e8dbc2b940 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.366973 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa3d2f-8dd1-4a4a-a6c2-42e8dbc2b940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.368793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-266e58b6-05bd-4aae-8ac7-ca8a1335a5f0 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:05:58.370133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-266e58b6-05bd-4aae-8ac7-ca8a1335a5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:05:58.375760 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:05:58.381289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 784/1554] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:05:58.383222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d617ae-c918-44c7-a890-965887aa66d9 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.384003 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d617ae-c918-44c7-a890-965887aa66d9 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.384797 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d617ae-c918-44c7-a890-965887aa66d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.391447 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-defa3d2f-8dd1-4a4a-a6c2-42e8dbc2b940 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:05:58.392866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-defa3d2f-8dd1-4a4a-a6c2-42e8dbc2b940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:05:58.401527 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 771/1555] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:05:58.402314 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa3d2f-8dd1-4a4a-a6c2-42e8dbc2b940 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:58.405909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e0d617ae-c918-44c7-a890-965887aa66d9 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:05:58.407283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e0d617ae-c918-44c7-a890-965887aa66d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:05:58.412595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa3d2f-8dd1-4a4a-a6c2-42e8dbc2b940 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:58.417073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 785/1556] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:05:58.417337 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cebde2-12d6-41a8-8d04-16c84a42e513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:58.418224 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cebde2-12d6-41a8-8d04-16c84a42e513 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.418349 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b643f-0905-46e6-b322-490dad61ce52 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.418533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cebde2-12d6-41a8-8d04-16c84a42e513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.418634 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b643f-0905-46e6-b322-490dad61ce52 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.418816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cebde2-12d6-41a8-8d04-16c84a42e513 admin admin] PATH_INFO: `/v3/projects/9bcee16336834b80a389799f89b5b79a/users/8a3137566b004edb909d4e3c702a7f59/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.418921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b643f-0905-46e6-b322-490dad61ce52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.438392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cebde2-12d6-41a8-8d04-16c84a42e513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bcee16336834b80a389799f89b5b79a, user_id=8a3137566b004edb909d4e3c702a7f59, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:58.439482 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: WARNING py.warnings [None req-d1cebde2-12d6-41a8-8d04-16c84a42e513 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 25 16:05:58.439482 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: warnings.warn(msg) Jun 25 16:05:58.439482 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: Jun 25 16:05:58.441468 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cebde2-12d6-41a8-8d04-16c84a42e513 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:58.441651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a96b643f-0905-46e6-b322-490dad61ce52 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:05:58.442850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a96b643f-0905-46e6-b322-490dad61ce52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:05:58.451360 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 786/1557] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:05:58.458504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78efd44-9a36-40fb-9214-637cd9416a5c None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.458504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78efd44-9a36-40fb-9214-637cd9416a5c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.458504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78efd44-9a36-40fb-9214-637cd9416a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.458801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 772/1558] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:05:58 2022] PUT /identity/v3/projects/9bcee16336834b80a389799f89b5b79a/users/8a3137566b004edb909d4e3c702a7f59/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:05:58.458801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cebde2-12d6-41a8-8d04-16c84a42e513 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:58.473570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b78efd44-9a36-40fb-9214-637cd9416a5c None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:05:58.474768 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b78efd44-9a36-40fb-9214-637cd9416a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:05:58.508291 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:05:58.522992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cebde2-12d6-41a8-8d04-16c84a42e513 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:58.528048 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74f97b-b3e6-442e-a05d-4d9dbbe28c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:58.528785 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74f97b-b3e6-442e-a05d-4d9dbbe28c47 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.529052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74f97b-b3e6-442e-a05d-4d9dbbe28c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.529439 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74f97b-b3e6-442e-a05d-4d9dbbe28c47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.530366 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b74f97b-b3e6-442e-a05d-4d9dbbe28c47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:05:58.530366 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b74f97b-b3e6-442e-a05d-4d9dbbe28c47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:58.532005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b74f97b-b3e6-442e-a05d-4d9dbbe28c47 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:58.538773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 787/1559] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:05:58.540284 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78efd44-9a36-40fb-9214-637cd9416a5c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:58.552545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 773/1560] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:05:58.552545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2dd25-8488-41b9-88f8-8ddef389adbd None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.552545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2dd25-8488-41b9-88f8-8ddef389adbd None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.552545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2dd25-8488-41b9-88f8-8ddef389adbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.558138 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78efd44-9a36-40fb-9214-637cd9416a5c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:58.562878 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff354d65-e9ee-4872-a27f-92ac46ae4507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:58.563579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff354d65-e9ee-4872-a27f-92ac46ae4507 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.563864 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff354d65-e9ee-4872-a27f-92ac46ae4507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.564139 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff354d65-e9ee-4872-a27f-92ac46ae4507 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.564897 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff354d65-e9ee-4872-a27f-92ac46ae4507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:58.565493 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a5a2dd25-8488-41b9-88f8-8ddef389adbd None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:05:58.566508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff354d65-e9ee-4872-a27f-92ac46ae4507 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:58.566706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a5a2dd25-8488-41b9-88f8-8ddef389adbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:05:58.574896 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 774/1561] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:05:58.576196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f3c48-e2df-48ed-a779-882fd52ba6e1 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.576511 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f3c48-e2df-48ed-a779-882fd52ba6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.576787 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f3c48-e2df-48ed-a779-882fd52ba6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.580022 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 788/1562] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:05:58.580022 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7810410-d515-4f60-978b-3367eeae37d2 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.580022 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7810410-d515-4f60-978b-3367eeae37d2 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.580022 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7810410-d515-4f60-978b-3367eeae37d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.596618 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-3a2f3c48-e2df-48ed-a779-882fd52ba6e1 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:05:58.597150 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c7810410-d515-4f60-978b-3367eeae37d2 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:05:58.597812 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3a2f3c48-e2df-48ed-a779-882fd52ba6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:05:58.598327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c7810410-d515-4f60-978b-3367eeae37d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:05:58.605745 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 775/1563] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:05:58.606953 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 789/1564] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:05:58.607086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cbf1e-67bb-4c1e-b1c9-3d66d294d5be None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.607251 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cbf1e-67bb-4c1e-b1c9-3d66d294d5be None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.607470 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cbf1e-67bb-4c1e-b1c9-3d66d294d5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.608077 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a70c09-a6f9-4899-8b6c-c4cfd7bb67a8 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.608367 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a70c09-a6f9-4899-8b6c-c4cfd7bb67a8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.608680 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a70c09-a6f9-4899-8b6c-c4cfd7bb67a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.625491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-414cbf1e-67bb-4c1e-b1c9-3d66d294d5be None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:05:58.626720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-414cbf1e-67bb-4c1e-b1c9-3d66d294d5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:05:58.627333 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c1a70c09-a6f9-4899-8b6c-c4cfd7bb67a8 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:05:58.629233 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c1a70c09-a6f9-4899-8b6c-c4cfd7bb67a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:05:58.638087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 776/1565] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:05:58.638087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90837b-290f-458d-84d4-0f0451684096 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.638087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90837b-290f-458d-84d4-0f0451684096 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.638087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90837b-290f-458d-84d4-0f0451684096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.642624 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 790/1566] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:05:58.643470 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a70c09-a6f9-4899-8b6c-c4cfd7bb67a8 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:58.653737 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a70c09-a6f9-4899-8b6c-c4cfd7bb67a8 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:58.658489 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-1f90837b-290f-458d-84d4-0f0451684096 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:05:58.658692 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173c3af-a674-45b4-b4c9-841154918c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:58.659152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173c3af-a674-45b4-b4c9-841154918c75 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.659434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173c3af-a674-45b4-b4c9-841154918c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.659711 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1173c3af-a674-45b4-b4c9-841154918c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.659806 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1f90837b-290f-458d-84d4-0f0451684096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:05:58.660426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1173c3af-a674-45b4-b4c9-841154918c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:58.662382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1173c3af-a674-45b4-b4c9-841154918c75 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:58.668240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 777/1567] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:05:58.669160 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90837b-290f-458d-84d4-0f0451684096 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:58.682742 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90837b-290f-458d-84d4-0f0451684096 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:58.688424 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a2c7b-717c-4094-86a9-c8f65cd4aaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:58.689123 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a2c7b-717c-4094-86a9-c8f65cd4aaeb admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.689236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 791/1568] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:05:58 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:05:58.689451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a2c7b-717c-4094-86a9-c8f65cd4aaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.689734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a2c7b-717c-4094-86a9-c8f65cd4aaeb admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.690031 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173c3af-a674-45b4-b4c9-841154918c75 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:58.690550 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a2c7b-717c-4094-86a9-c8f65cd4aaeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:05:58.690965 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a2c7b-717c-4094-86a9-c8f65cd4aaeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:58.692950 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a2c7b-717c-4094-86a9-c8f65cd4aaeb admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:58.706109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 778/1569] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:05:58.707085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a2c7b-717c-4094-86a9-c8f65cd4aaeb admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:58.722932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a2c7b-717c-4094-86a9-c8f65cd4aaeb admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:58.755735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1173c3af-a674-45b4-b4c9-841154918c75 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:58.760880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b8b68-d4da-4cf6-8376-26c1016f8ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:58.761485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b8b68-d4da-4cf6-8376-26c1016f8ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.761767 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b8b68-d4da-4cf6-8376-26c1016f8ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.762039 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b8b68-d4da-4cf6-8376-26c1016f8ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.762763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b8b68-d4da-4cf6-8376-26c1016f8ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:58.764356 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b8b68-d4da-4cf6-8376-26c1016f8ef2 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:58.775336 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 792/1570] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:05:58.776172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b8b68-d4da-4cf6-8376-26c1016f8ef2 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:58.780312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979666ec-a60e-4676-a98b-389ce20379ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:58.781014 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979666ec-a60e-4676-a98b-389ce20379ce admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.781295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979666ec-a60e-4676-a98b-389ce20379ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.781566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979666ec-a60e-4676-a98b-389ce20379ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.782304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979666ec-a60e-4676-a98b-389ce20379ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:05:58.782607 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979666ec-a60e-4676-a98b-389ce20379ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:58.784265 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979666ec-a60e-4676-a98b-389ce20379ce admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:58.785961 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b8b68-d4da-4cf6-8376-26c1016f8ef2 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:58.791044 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3afb49-3844-4fd0-a73c-ee43bf36caf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:58.791663 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3afb49-3844-4fd0-a73c-ee43bf36caf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.791942 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3afb49-3844-4fd0-a73c-ee43bf36caf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.792216 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3afb49-3844-4fd0-a73c-ee43bf36caf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.792969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3afb49-3844-4fd0-a73c-ee43bf36caf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:05:58.793279 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3afb49-3844-4fd0-a73c-ee43bf36caf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:58.794878 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3afb49-3844-4fd0-a73c-ee43bf36caf0 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:58.803711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 779/1571] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:05:58.803711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979666ec-a60e-4676-a98b-389ce20379ce admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:58.807556 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979666ec-a60e-4676-a98b-389ce20379ce admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:58.813684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 793/1572] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:05:58.813684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3afb49-3844-4fd0-a73c-ee43bf36caf0 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:58.815166 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a08037-a84b-4c62-8094-a7c10d6cecd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:58.815830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a08037-a84b-4c62-8094-a7c10d6cecd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.816369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a08037-a84b-4c62-8094-a7c10d6cecd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.816676 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a08037-a84b-4c62-8094-a7c10d6cecd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.817410 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a08037-a84b-4c62-8094-a7c10d6cecd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:05:58.817713 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a08037-a84b-4c62-8094-a7c10d6cecd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:58.819344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a08037-a84b-4c62-8094-a7c10d6cecd0 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:58.832150 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 780/1573] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:05:58.832150 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a08037-a84b-4c62-8094-a7c10d6cecd0 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:58.849164 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a08037-a84b-4c62-8094-a7c10d6cecd0 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:58.854080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e73ad-741d-4f01-9bf6-1d1c0680b8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:58.854743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e73ad-741d-4f01-9bf6-1d1c0680b8c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.855021 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e73ad-741d-4f01-9bf6-1d1c0680b8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.855286 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e73ad-741d-4f01-9bf6-1d1c0680b8c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.856030 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e73ad-741d-4f01-9bf6-1d1c0680b8c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:05:58.856466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e73ad-741d-4f01-9bf6-1d1c0680b8c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:58.857981 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e73ad-741d-4f01-9bf6-1d1c0680b8c9 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:58.860638 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:05:58.871503 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 781/1574] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:05:58.876722 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e73ad-741d-4f01-9bf6-1d1c0680b8c9 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:58.881126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3afb49-3844-4fd0-a73c-ee43bf36caf0 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:58.885021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa5ed6-189a-4176-98eb-a1f8ec99fece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:58.885667 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa5ed6-189a-4176-98eb-a1f8ec99fece service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.886078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa5ed6-189a-4176-98eb-a1f8ec99fece service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.886458 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa5ed6-189a-4176-98eb-a1f8ec99fece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.887430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e73ad-741d-4f01-9bf6-1d1c0680b8c9 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:58.892530 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7d19e-55b7-4440-b8ca-a615f6a30a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:58.893175 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7d19e-55b7-4440-b8ca-a615f6a30a80 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.893450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7d19e-55b7-4440-b8ca-a615f6a30a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.894147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7d19e-55b7-4440-b8ca-a615f6a30a80 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.894390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7d19e-55b7-4440-b8ca-a615f6a30a80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:58.895994 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa5ed6-189a-4176-98eb-a1f8ec99fece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:58.896285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7d19e-55b7-4440-b8ca-a615f6a30a80 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:58.897457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa5ed6-189a-4176-98eb-a1f8ec99fece service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:58.907472 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 794/1575] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:05:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:05:58.908291 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa5ed6-189a-4176-98eb-a1f8ec99fece service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:58.920170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa5ed6-189a-4176-98eb-a1f8ec99fece service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:58.923492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 782/1576] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:05:58 2022] POST /identity/v3/users => generated 283 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:05:58.923492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7d19e-55b7-4440-b8ca-a615f6a30a80 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:58.925369 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795a515-67d7-4395-80aa-c4a620afe49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:58.926029 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795a515-67d7-4395-80aa-c4a620afe49e admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.926306 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795a515-67d7-4395-80aa-c4a620afe49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.926581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795a515-67d7-4395-80aa-c4a620afe49e admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.927262 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795a515-67d7-4395-80aa-c4a620afe49e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:58.929241 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795a515-67d7-4395-80aa-c4a620afe49e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:58.932211 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7d19e-55b7-4440-b8ca-a615f6a30a80 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:58.937759 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967338be-3a66-4fd1-a85c-416b06347728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:58.938661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967338be-3a66-4fd1-a85c-416b06347728 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:58.938942 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967338be-3a66-4fd1-a85c-416b06347728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:58.939229 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967338be-3a66-4fd1-a85c-416b06347728 admin admin] PATH_INFO: `/v3/projects/d14e0266d46e4e8ebc64a36f20766c6a/users/ccc3c8fcd64b4b618d99a092228dd345/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:58.945586 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:05:58.955492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967338be-3a66-4fd1-a85c-416b06347728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d14e0266d46e4e8ebc64a36f20766c6a, user_id=ccc3c8fcd64b4b618d99a092228dd345, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:58.958186 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967338be-3a66-4fd1-a85c-416b06347728 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:58.960464 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 795/1577] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:05:58 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:05:58.961792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795a515-67d7-4395-80aa-c4a620afe49e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:58.976818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 783/1578] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:05:58 2022] PUT /identity/v3/projects/d14e0266d46e4e8ebc64a36f20766c6a/users/ccc3c8fcd64b4b618d99a092228dd345/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:05:58.977994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967338be-3a66-4fd1-a85c-416b06347728 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.041722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795a515-67d7-4395-80aa-c4a620afe49e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.046486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f12fe-1558-47e9-ae48-1fbeccad6a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.047172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f12fe-1558-47e9-ae48-1fbeccad6a2f admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.047445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f12fe-1558-47e9-ae48-1fbeccad6a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.047726 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f12fe-1558-47e9-ae48-1fbeccad6a2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.048454 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f12fe-1558-47e9-ae48-1fbeccad6a2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.050365 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f12fe-1558-47e9-ae48-1fbeccad6a2f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.051274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967338be-3a66-4fd1-a85c-416b06347728 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.057249 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3bdfa-ed47-4df9-9dab-eac3e21df69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.058144 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3bdfa-ed47-4df9-9dab-eac3e21df69a admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.058428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3bdfa-ed47-4df9-9dab-eac3e21df69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.058699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3bdfa-ed47-4df9-9dab-eac3e21df69a admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.059748 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3bdfa-ed47-4df9-9dab-eac3e21df69a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.062196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3bdfa-ed47-4df9-9dab-eac3e21df69a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.082388 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 796/1579] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:05:58 2022] POST /identity/v3/projects => generated 405 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:05:59.083911 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f12fe-1558-47e9-ae48-1fbeccad6a2f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.094187 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 784/1580] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:05:58 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:05:59.095320 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3bdfa-ed47-4df9-9dab-eac3e21df69a admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.168085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f12fe-1558-47e9-ae48-1fbeccad6a2f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.172142 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3bdfa-ed47-4df9-9dab-eac3e21df69a admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.174179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee98ed-1687-47c4-a502-91d4595085ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.174860 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee98ed-1687-47c4-a502-91d4595085ae admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.175145 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee98ed-1687-47c4-a502-91d4595085ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.175428 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deee98ed-1687-47c4-a502-91d4595085ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.176130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee98ed-1687-47c4-a502-91d4595085ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.177457 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cbee9-198e-4266-b275-f98f7c5d337a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.178197 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deee98ed-1687-47c4-a502-91d4595085ae admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.178306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cbee9-198e-4266-b275-f98f7c5d337a admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.178536 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cbee9-198e-4266-b275-f98f7c5d337a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.178873 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cbee9-198e-4266-b275-f98f7c5d337a admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.179827 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6cbee9-198e-4266-b275-f98f7c5d337a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.182245 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6cbee9-198e-4266-b275-f98f7c5d337a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.209925 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 797/1581] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:05:59 2022] POST /identity/v3/projects => generated 363 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:05:59.210598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee98ed-1687-47c4-a502-91d4595085ae admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.221551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 785/1582] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:05:59 2022] POST /identity/v3/projects => generated 379 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:05:59.222318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cbee9-198e-4266-b275-f98f7c5d337a admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.255653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:05:59.293341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deee98ed-1687-47c4-a502-91d4595085ae admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.296595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cbee9-198e-4266-b275-f98f7c5d337a admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.298678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60064564-b1ab-4f7f-8dfe-f01211a3c643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.299552 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60064564-b1ab-4f7f-8dfe-f01211a3c643 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.299977 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60064564-b1ab-4f7f-8dfe-f01211a3c643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.301016 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60064564-b1ab-4f7f-8dfe-f01211a3c643 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.301518 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79199179-7c5a-43bd-a455-b585568a77d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.302100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60064564-b1ab-4f7f-8dfe-f01211a3c643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.302342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79199179-7c5a-43bd-a455-b585568a77d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.302690 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79199179-7c5a-43bd-a455-b585568a77d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.303033 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79199179-7c5a-43bd-a455-b585568a77d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.303797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79199179-7c5a-43bd-a455-b585568a77d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.304658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60064564-b1ab-4f7f-8dfe-f01211a3c643 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.306463 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79199179-7c5a-43bd-a455-b585568a77d6 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.318085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 798/1583] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:05:59.319478 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d3f90-fac5-42b6-9d64-4426f4b1e91d None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.319763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d3f90-fac5-42b6-9d64-4426f4b1e91d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.320060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d3f90-fac5-42b6-9d64-4426f4b1e91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.327008 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:05:59.335238 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 786/1584] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:05:59 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:05:59.339101 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79199179-7c5a-43bd-a455-b585568a77d6 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.341231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-4c2d3f90-fac5-42b6-9d64-4426f4b1e91d None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:05:59.343088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4c2d3f90-fac5-42b6-9d64-4426f4b1e91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:05:59.349959 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79199179-7c5a-43bd-a455-b585568a77d6 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.351422 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 799/1585] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:05:59.352184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d3f90-fac5-42b6-9d64-4426f4b1e91d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.355737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b230a-14bd-4093-90bf-827b01fcbfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.356680 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b230a-14bd-4093-90bf-827b01fcbfd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.357125 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b230a-14bd-4093-90bf-827b01fcbfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.357408 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b230a-14bd-4093-90bf-827b01fcbfd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.358663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b230a-14bd-4093-90bf-827b01fcbfd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.361083 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b230a-14bd-4093-90bf-827b01fcbfd9 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.362699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d3f90-fac5-42b6-9d64-4426f4b1e91d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.367173 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e581b-7d05-4541-a15f-9274b885be11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.368117 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e581b-7d05-4541-a15f-9274b885be11 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.368281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e581b-7d05-4541-a15f-9274b885be11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.368617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e581b-7d05-4541-a15f-9274b885be11 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.369645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e581b-7d05-4541-a15f-9274b885be11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.370511 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: WARNING py.warnings [None req-824e581b-7d05-4541-a15f-9274b885be11 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 25 16:05:59.370511 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: warnings.warn(msg) Jun 25 16:05:59.370511 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: Jun 25 16:05:59.371795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e581b-7d05-4541-a15f-9274b885be11 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.396645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 800/1586] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:05:59 2022] POST /identity/v3/users => generated 292 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:05:59.397570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e581b-7d05-4541-a15f-9274b885be11 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.403002 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 787/1587] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:05:59 2022] POST /identity/v3/users => generated 303 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:05:59.403002 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b230a-14bd-4093-90bf-827b01fcbfd9 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.410517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e581b-7d05-4541-a15f-9274b885be11 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.416297 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a43c6-8c82-4994-b633-b4a35b5d6041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.417359 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a43c6-8c82-4994-b633-b4a35b5d6041 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.417713 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a43c6-8c82-4994-b633-b4a35b5d6041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.418086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a43c6-8c82-4994-b633-b4a35b5d6041 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.418886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a43c6-8c82-4994-b633-b4a35b5d6041 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.419776 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b230a-14bd-4093-90bf-827b01fcbfd9 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.421409 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a43c6-8c82-4994-b633-b4a35b5d6041 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.425020 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901de3f-9ce8-4afb-8b70-be5288a26bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.425696 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901de3f-9ce8-4afb-8b70-be5288a26bde admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.425978 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901de3f-9ce8-4afb-8b70-be5288a26bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.426255 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901de3f-9ce8-4afb-8b70-be5288a26bde admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.426947 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901de3f-9ce8-4afb-8b70-be5288a26bde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.432091 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901de3f-9ce8-4afb-8b70-be5288a26bde admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.456838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 801/1588] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:05:59 2022] POST /identity/v3/users => generated 282 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:05:59.456838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a43c6-8c82-4994-b633-b4a35b5d6041 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.466885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 788/1589] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:05:59 2022] POST /identity/v3/users => generated 290 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:05:59.466885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901de3f-9ce8-4afb-8b70-be5288a26bde admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.468300 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a43c6-8c82-4994-b633-b4a35b5d6041 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.473373 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa67e2e-b90f-4554-8fd7-98918734ce30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.474090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa67e2e-b90f-4554-8fd7-98918734ce30 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.474372 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa67e2e-b90f-4554-8fd7-98918734ce30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.474647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa67e2e-b90f-4554-8fd7-98918734ce30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.475392 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa67e2e-b90f-4554-8fd7-98918734ce30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.477059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa67e2e-b90f-4554-8fd7-98918734ce30 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.477671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901de3f-9ce8-4afb-8b70-be5288a26bde admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.483497 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee433ebd-68ea-43c0-ac3a-9d28a86661f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.484174 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee433ebd-68ea-43c0-ac3a-9d28a86661f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.484486 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee433ebd-68ea-43c0-ac3a-9d28a86661f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.485046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee433ebd-68ea-43c0-ac3a-9d28a86661f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.485782 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee433ebd-68ea-43c0-ac3a-9d28a86661f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.487573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee433ebd-68ea-43c0-ac3a-9d28a86661f6 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.489079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 802/1590] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:05:59.489422 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa67e2e-b90f-4554-8fd7-98918734ce30 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.491522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-caa67e2e-b90f-4554-8fd7-98918734ce30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:05:59.503319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 789/1591] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:05:59.503319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee433ebd-68ea-43c0-ac3a-9d28a86661f6 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.512890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee433ebd-68ea-43c0-ac3a-9d28a86661f6 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.518145 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39caf4e7-00d7-472f-b974-3e901274fbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.518816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39caf4e7-00d7-472f-b974-3e901274fbb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.519099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39caf4e7-00d7-472f-b974-3e901274fbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.519372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39caf4e7-00d7-472f-b974-3e901274fbb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.520196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39caf4e7-00d7-472f-b974-3e901274fbb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.521843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39caf4e7-00d7-472f-b974-3e901274fbb9 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.534027 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 790/1592] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:05:59.535303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39caf4e7-00d7-472f-b974-3e901274fbb9 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.547679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39caf4e7-00d7-472f-b974-3e901274fbb9 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.552906 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e161cb0-12f4-4718-b272-760a346621e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.553599 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e161cb0-12f4-4718-b272-760a346621e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.553895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e161cb0-12f4-4718-b272-760a346621e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.554171 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e161cb0-12f4-4718-b272-760a346621e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.554918 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e161cb0-12f4-4718-b272-760a346621e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.556674 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e161cb0-12f4-4718-b272-760a346621e9 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.563383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa67e2e-b90f-4554-8fd7-98918734ce30 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.568645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86335c6-7681-4311-a34d-6312edaf4a36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.568831 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 791/1593] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:05:59.569469 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86335c6-7681-4311-a34d-6312edaf4a36 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.569589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e161cb0-12f4-4718-b272-760a346621e9 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.569850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86335c6-7681-4311-a34d-6312edaf4a36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.570134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86335c6-7681-4311-a34d-6312edaf4a36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.582852 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86335c6-7681-4311-a34d-6312edaf4a36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.583093 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e161cb0-12f4-4718-b272-760a346621e9 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.584303 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86335c6-7681-4311-a34d-6312edaf4a36 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.588258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9600a0-66cf-4e87-a89c-f36fc9615494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.589000 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9600a0-66cf-4e87-a89c-f36fc9615494 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.589277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9600a0-66cf-4e87-a89c-f36fc9615494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.589548 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9600a0-66cf-4e87-a89c-f36fc9615494 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.590272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9600a0-66cf-4e87-a89c-f36fc9615494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.591879 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9600a0-66cf-4e87-a89c-f36fc9615494 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.595707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 803/1594] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:05:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:05:59.596645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86335c6-7681-4311-a34d-6312edaf4a36 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.607286 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 792/1595] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:05:59.607286 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9600a0-66cf-4e87-a89c-f36fc9615494 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.607658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86335c6-7681-4311-a34d-6312edaf4a36 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.613473 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f697e-85a9-4444-9bca-c9d9c19d257b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.614157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f697e-85a9-4444-9bca-c9d9c19d257b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.614563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f697e-85a9-4444-9bca-c9d9c19d257b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.614889 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f697e-85a9-4444-9bca-c9d9c19d257b admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.615114 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9600a0-66cf-4e87-a89c-f36fc9615494 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.615628 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f697e-85a9-4444-9bca-c9d9c19d257b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.617310 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f697e-85a9-4444-9bca-c9d9c19d257b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.621164 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ec9b6-4282-42c5-9964-110fd06f5d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.622294 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ec9b6-4282-42c5-9964-110fd06f5d66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.622708 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ec9b6-4282-42c5-9964-110fd06f5d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.623004 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ec9b6-4282-42c5-9964-110fd06f5d66 admin admin] PATH_INFO: `/v3/projects/3afb1695c99040d9a9812815a5e7b419/users/755c0268c22a479d983dba06c4122a90/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.639196 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ec9b6-4282-42c5-9964-110fd06f5d66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3afb1695c99040d9a9812815a5e7b419, user_id=755c0268c22a479d983dba06c4122a90, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.641223 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 804/1596] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:05:59.642332 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ec9b6-4282-42c5-9964-110fd06f5d66 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.642458 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f697e-85a9-4444-9bca-c9d9c19d257b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.656275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f697e-85a9-4444-9bca-c9d9c19d257b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.666814 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4905a1b-f142-4750-8dcf-9689bc0ab05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.667499 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4905a1b-f142-4750-8dcf-9689bc0ab05b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.667783 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4905a1b-f142-4750-8dcf-9689bc0ab05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.668060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4905a1b-f142-4750-8dcf-9689bc0ab05b admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.668903 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4905a1b-f142-4750-8dcf-9689bc0ab05b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.670494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4905a1b-f142-4750-8dcf-9689bc0ab05b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.672505 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 793/1597] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:05:59 2022] PUT /identity/v3/projects/3afb1695c99040d9a9812815a5e7b419/users/755c0268c22a479d983dba06c4122a90/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:05:59.673542 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ec9b6-4282-42c5-9964-110fd06f5d66 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.696461 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 805/1598] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:05:59.698726 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4905a1b-f142-4750-8dcf-9689bc0ab05b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.716364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4905a1b-f142-4750-8dcf-9689bc0ab05b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.778119 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ec9b6-4282-42c5-9964-110fd06f5d66 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.785172 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87321460-5c2e-439e-b9f3-83f9ba13c38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.786008 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87321460-5c2e-439e-b9f3-83f9ba13c38e admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.786818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87321460-5c2e-439e-b9f3-83f9ba13c38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.787105 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87321460-5c2e-439e-b9f3-83f9ba13c38e admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.788312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87321460-5c2e-439e-b9f3-83f9ba13c38e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.792119 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87321460-5c2e-439e-b9f3-83f9ba13c38e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.809109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 794/1599] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:05:59.810493 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87321460-5c2e-439e-b9f3-83f9ba13c38e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.823020 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87321460-5c2e-439e-b9f3-83f9ba13c38e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.828277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0dd39-cafe-471a-81f8-23c654f0b607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.829240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0dd39-cafe-471a-81f8-23c654f0b607 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.829530 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0dd39-cafe-471a-81f8-23c654f0b607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.829825 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0dd39-cafe-471a-81f8-23c654f0b607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.830513 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0dd39-cafe-471a-81f8-23c654f0b607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.832315 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0dd39-cafe-471a-81f8-23c654f0b607 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.833996 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:05:59.845731 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 795/1600] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:05:59.846802 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0dd39-cafe-471a-81f8-23c654f0b607 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.865135 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0dd39-cafe-471a-81f8-23c654f0b607 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.865849 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20636e-a7b9-4014-91de-60514d3f6e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.866659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20636e-a7b9-4014-91de-60514d3f6e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.867027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20636e-a7b9-4014-91de-60514d3f6e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.867506 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20636e-a7b9-4014-91de-60514d3f6e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.868879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20636e-a7b9-4014-91de-60514d3f6e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.872242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa72078-f8e2-472d-b27f-9259aa3833ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.872928 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20636e-a7b9-4014-91de-60514d3f6e12 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.873496 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa72078-f8e2-472d-b27f-9259aa3833ae admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.873976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa72078-f8e2-472d-b27f-9259aa3833ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.874420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa72078-f8e2-472d-b27f-9259aa3833ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.875453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa72078-f8e2-472d-b27f-9259aa3833ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.878189 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa72078-f8e2-472d-b27f-9259aa3833ae admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.889692 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 806/1601] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:05:59.896368 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a1f72-f5a1-4154-a3fc-607ac8e46b08 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.896682 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a1f72-f5a1-4154-a3fc-607ac8e46b08 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.896941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a1f72-f5a1-4154-a3fc-607ac8e46b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.897908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 796/1602] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:05:59.899753 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa72078-f8e2-472d-b27f-9259aa3833ae admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.916049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa72078-f8e2-472d-b27f-9259aa3833ae admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:05:59.922625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ec7c4-f742-4482-ae2f-f5089b437c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:05:59.925185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ec7c4-f742-4482-ae2f-f5089b437c4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:05:59.925773 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ec7c4-f742-4482-ae2f-f5089b437c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:05:59.926325 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ec7c4-f742-4482-ae2f-f5089b437c4c admin admin] PATH_INFO: `/v3/projects/b1bba8f74dbd433bbffdb2ffbfc481fb/users/19140330ae664e45b85f47f321496191/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:05:59.948807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ec7c4-f742-4482-ae2f-f5089b437c4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1bba8f74dbd433bbffdb2ffbfc481fb, user_id=19140330ae664e45b85f47f321496191, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:05:59.953049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ec7c4-f742-4482-ae2f-f5089b437c4c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:05:59.973503 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 797/1603] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:05:59 2022] PUT /identity/v3/projects/b1bba8f74dbd433bbffdb2ffbfc481fb/users/19140330ae664e45b85f47f321496191/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:05:59.974516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ec7c4-f742-4482-ae2f-f5089b437c4c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:05:59.976184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-816a1f72-f5a1-4154-a3fc-607ac8e46b08 None None] MFA Rules not processed for user `755c0268c22a479d983dba06c4122a90`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:05:59.979369 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-816a1f72-f5a1-4154-a3fc-607ac8e46b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:00.013913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:00.100925 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ec7c4-f742-4482-ae2f-f5089b437c4c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:00.107660 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f8d5f-93a2-4d5b-ac9a-4c07acf121c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:00.109647 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f8d5f-93a2-4d5b-ac9a-4c07acf121c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:00.109908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f8d5f-93a2-4d5b-ac9a-4c07acf121c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:00.110363 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f8d5f-93a2-4d5b-ac9a-4c07acf121c2 admin admin] PATH_INFO: `/v3/projects/8ef564f495b4489e98a852ad9fb7c5bd/users/33533f18b16f48e08babc4c07956a0d6/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:00.125102 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 807/1604] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:05:59 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:00.125952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a1f72-f5a1-4154-a3fc-607ac8e46b08 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:00.141391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f8d5f-93a2-4d5b-ac9a-4c07acf121c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ef564f495b4489e98a852ad9fb7c5bd, user_id=33533f18b16f48e08babc4c07956a0d6, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:00.145519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f8d5f-93a2-4d5b-ac9a-4c07acf121c2 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:00.153354 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a1f72-f5a1-4154-a3fc-607ac8e46b08 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:00.165138 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367e36d-f5ac-48eb-a464-e71b4c2d707d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:00.166132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367e36d-f5ac-48eb-a464-e71b4c2d707d admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:00.166412 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367e36d-f5ac-48eb-a464-e71b4c2d707d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:00.166688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367e36d-f5ac-48eb-a464-e71b4c2d707d admin admin] PATH_INFO: `/v3/projects/6c08311d50d54d4ead8b533530ae3dc9/users/def03b41c32941778ab452bd83982949/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:00.168043 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 798/1605] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:05:59 2022] PUT /identity/v3/projects/8ef564f495b4489e98a852ad9fb7c5bd/users/33533f18b16f48e08babc4c07956a0d6/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:00.172785 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f8d5f-93a2-4d5b-ac9a-4c07acf121c2 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:00.200085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367e36d-f5ac-48eb-a464-e71b4c2d707d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c08311d50d54d4ead8b533530ae3dc9, user_id=def03b41c32941778ab452bd83982949, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:00.203339 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367e36d-f5ac-48eb-a464-e71b4c2d707d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:00.222989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 808/1606] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:00 2022] PUT /identity/v3/projects/6c08311d50d54d4ead8b533530ae3dc9/users/def03b41c32941778ab452bd83982949/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:00.224195 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367e36d-f5ac-48eb-a464-e71b4c2d707d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:00.279029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f8d5f-93a2-4d5b-ac9a-4c07acf121c2 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:00.330591 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:00.347915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91292c6b-8288-4a8a-a3ce-8a5b937765ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:00.349013 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91292c6b-8288-4a8a-a3ce-8a5b937765ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:00.349593 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91292c6b-8288-4a8a-a3ce-8a5b937765ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:00.349886 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91292c6b-8288-4a8a-a3ce-8a5b937765ec admin admin] PATH_INFO: `/v3/projects/0b4842ea7c7a463598d269c34e16c7ba/users/a069e0ece7974f0b9072be533518980a/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:00.366158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367e36d-f5ac-48eb-a464-e71b4c2d707d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:00.373080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7ec9d-d027-4ae3-b2ef-41b2f31eab2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:00.374208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7ec9d-d027-4ae3-b2ef-41b2f31eab2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:00.374491 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7ec9d-d027-4ae3-b2ef-41b2f31eab2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:00.374768 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7ec9d-d027-4ae3-b2ef-41b2f31eab2f admin admin] PATH_INFO: `/v3/projects/c348649c6b794f5c9339ea9506b6a16b/users/be06179595ef450a917d71fe4e6b2c5b/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:00.384096 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91292c6b-8288-4a8a-a3ce-8a5b937765ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b4842ea7c7a463598d269c34e16c7ba, user_id=a069e0ece7974f0b9072be533518980a, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:00.387505 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91292c6b-8288-4a8a-a3ce-8a5b937765ec admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:00.404709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7ec9d-d027-4ae3-b2ef-41b2f31eab2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c348649c6b794f5c9339ea9506b6a16b, user_id=be06179595ef450a917d71fe4e6b2c5b, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:00.407901 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7ec9d-d027-4ae3-b2ef-41b2f31eab2f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:00.412265 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 799/1607] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:00 2022] PUT /identity/v3/projects/0b4842ea7c7a463598d269c34e16c7ba/users/a069e0ece7974f0b9072be533518980a/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:00.416816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a5506-0118-4d8a-a2f4-1a050cef5bdb None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:00.417389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a5506-0118-4d8a-a2f4-1a050cef5bdb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:00.417915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a5506-0118-4d8a-a2f4-1a050cef5bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:00.443893 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 809/1608] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:00 2022] PUT /identity/v3/projects/c348649c6b794f5c9339ea9506b6a16b/users/be06179595ef450a917d71fe4e6b2c5b/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:00.445138 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7ec9d-d027-4ae3-b2ef-41b2f31eab2f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:00.452587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ae3a5506-0118-4d8a-a2f4-1a050cef5bdb None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:00.453877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ae3a5506-0118-4d8a-a2f4-1a050cef5bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:00.572929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 800/1609] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:00.576734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc8b69-ee9d-44ab-8088-f61e1ebcded9 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:00.577030 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc8b69-ee9d-44ab-8088-f61e1ebcded9 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:00.577309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc8b69-ee9d-44ab-8088-f61e1ebcded9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:00.580784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7ec9d-d027-4ae3-b2ef-41b2f31eab2f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:00.595231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db515b-ba25-4fc0-83bd-f4e353f33b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:00.595916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db515b-ba25-4fc0-83bd-f4e353f33b26 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:00.596198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db515b-ba25-4fc0-83bd-f4e353f33b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:00.596507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db515b-ba25-4fc0-83bd-f4e353f33b26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:00.597229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db515b-ba25-4fc0-83bd-f4e353f33b26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:00.599153 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db515b-ba25-4fc0-83bd-f4e353f33b26 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:00.626707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c4cc8b69-ee9d-44ab-8088-f61e1ebcded9 None None] MFA Rules not processed for user `33533f18b16f48e08babc4c07956a0d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:00.629370 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c4cc8b69-ee9d-44ab-8088-f61e1ebcded9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:00.644762 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:00.662293 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 810/1610] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:00 2022] POST /identity/v3/projects => generated 361 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:00.664214 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d835401-b51f-45ae-98d5-fbacb959915c None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:00.665125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d835401-b51f-45ae-98d5-fbacb959915c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:00.665551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d835401-b51f-45ae-98d5-fbacb959915c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:00.706063 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3d835401-b51f-45ae-98d5-fbacb959915c None None] MFA Rules not processed for user `def03b41c32941778ab452bd83982949`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:00.707626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3d835401-b51f-45ae-98d5-fbacb959915c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:00.756502 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 801/1611] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:00.783047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d21d1-3b68-4b15-bc3f-ae509488f560 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:00.784201 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d21d1-3b68-4b15-bc3f-ae509488f560 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:00.785054 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d21d1-3b68-4b15-bc3f-ae509488f560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:00.829677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 811/1612] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:00 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:00.830986 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4496297-ef8a-4df2-8b5e-c2c9f5d097d4 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:00.831349 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4496297-ef8a-4df2-8b5e-c2c9f5d097d4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:00.831764 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4496297-ef8a-4df2-8b5e-c2c9f5d097d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:00.854022 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-d4496297-ef8a-4df2-8b5e-c2c9f5d097d4 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:00.855786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d4496297-ef8a-4df2-8b5e-c2c9f5d097d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:00.865137 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-285d21d1-3b68-4b15-bc3f-ae509488f560 None None] MFA Rules not processed for user `a069e0ece7974f0b9072be533518980a`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:00.869619 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-285d21d1-3b68-4b15-bc3f-ae509488f560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:00.959955 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 812/1613] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:00.961007 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4496297-ef8a-4df2-8b5e-c2c9f5d097d4 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:00.969982 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:00.998313 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 802/1614] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:00 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:00.998973 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d21d1-3b68-4b15-bc3f-ae509488f560 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:01.029657 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d21d1-3b68-4b15-bc3f-ae509488f560 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:01.042907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffcc494-f2e5-4861-a195-3c2bca5a8017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:01.043612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcc494-f2e5-4861-a195-3c2bca5a8017 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:01.044028 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcc494-f2e5-4861-a195-3c2bca5a8017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:01.044215 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcc494-f2e5-4861-a195-3c2bca5a8017 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:01.044958 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffcc494-f2e5-4861-a195-3c2bca5a8017 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:01.046929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffcc494-f2e5-4861-a195-3c2bca5a8017 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:01.058159 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4496297-ef8a-4df2-8b5e-c2c9f5d097d4 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:01.062345 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff0122-4344-4131-9d06-76211753d397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:01.063084 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff0122-4344-4131-9d06-76211753d397 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:01.063575 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff0122-4344-4131-9d06-76211753d397 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:01.064313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff0122-4344-4131-9d06-76211753d397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:01.079915 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff0122-4344-4131-9d06-76211753d397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:01.081892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff0122-4344-4131-9d06-76211753d397 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:01.096390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 813/1615] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:01.098278 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08148d-a18d-4dcf-9b80-bf4f735e1d6f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:01.098634 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08148d-a18d-4dcf-9b80-bf4f735e1d6f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:01.099178 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08148d-a18d-4dcf-9b80-bf4f735e1d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:01.104655 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 803/1616] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:00 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:01.109989 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcba29b-c1b8-4e99-8a79-823e93ce815b None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:01.110969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcba29b-c1b8-4e99-8a79-823e93ce815b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:01.111256 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcba29b-c1b8-4e99-8a79-823e93ce815b None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:01.113886 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 804/1617] 10.222.0.13 () {54 vars in 961 bytes} [Sat Jun 25 16:06:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:06:01.114530 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcba29b-c1b8-4e99-8a79-823e93ce815b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:01.130224 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcba29b-c1b8-4e99-8a79-823e93ce815b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:01.130997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8a08148d-a18d-4dcf-9b80-bf4f735e1d6f None None] MFA Rules not processed for user `ccc3c8fcd64b4b618d99a092228dd345`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:01.133175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8a08148d-a18d-4dcf-9b80-bf4f735e1d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:01.136774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd90507-5794-445d-8390-3c80199b9b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:01.137711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd90507-5794-445d-8390-3c80199b9b16 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:01.138187 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd90507-5794-445d-8390-3c80199b9b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:01.138287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd90507-5794-445d-8390-3c80199b9b16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:01.138977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd90507-5794-445d-8390-3c80199b9b16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:01.144225 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd90507-5794-445d-8390-3c80199b9b16 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:01.158395 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:01.184949 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 805/1618] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:01 2022] POST /identity/v3/projects => generated 365 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:01.186440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd90507-5794-445d-8390-3c80199b9b16 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:01.274483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 814/1619] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:01.274483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08148d-a18d-4dcf-9b80-bf4f735e1d6f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:01.274483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08148d-a18d-4dcf-9b80-bf4f735e1d6f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:01.300395 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd90507-5794-445d-8390-3c80199b9b16 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:01.305801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee2067-d5de-4d02-987e-a054049adf36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:01.307131 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee2067-d5de-4d02-987e-a054049adf36 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:01.307949 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee2067-d5de-4d02-987e-a054049adf36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:01.308482 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee2067-d5de-4d02-987e-a054049adf36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:01.348840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e0043-58df-4bae-82fd-ba9268ac73f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:01.349522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e0043-58df-4bae-82fd-ba9268ac73f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:01.349807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e0043-58df-4bae-82fd-ba9268ac73f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:01.350345 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e0043-58df-4bae-82fd-ba9268ac73f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:01.350776 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e0043-58df-4bae-82fd-ba9268ac73f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:01.353405 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e0043-58df-4bae-82fd-ba9268ac73f9 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:01.385926 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 815/1620] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:01 2022] POST /identity/v3/projects => generated 405 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:01.385926 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e0043-58df-4bae-82fd-ba9268ac73f9 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:01.414470 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee2067-d5de-4d02-987e-a054049adf36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:01.418654 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee2067-d5de-4d02-987e-a054049adf36 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:01.432825 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 806/1621] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:01.438062 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74512c42-6b15-4232-927b-a617cf94f48b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:01.438586 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74512c42-6b15-4232-927b-a617cf94f48b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:01.438872 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74512c42-6b15-4232-927b-a617cf94f48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:01.462520 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-74512c42-6b15-4232-927b-a617cf94f48b None None] MFA Rules not processed for user `3622c5ccb7534e7c8e8342b4f6643bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:01.464180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-74512c42-6b15-4232-927b-a617cf94f48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:01.476253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e0043-58df-4bae-82fd-ba9268ac73f9 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:01.481507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631c9e3-8995-49b8-818b-392b6796ae21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:01.482513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631c9e3-8995-49b8-818b-392b6796ae21 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:01.482723 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631c9e3-8995-49b8-818b-392b6796ae21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:01.482935 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631c9e3-8995-49b8-818b-392b6796ae21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:01.483968 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631c9e3-8995-49b8-818b-392b6796ae21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:01.485631 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631c9e3-8995-49b8-818b-392b6796ae21 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:01.497740 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 816/1622] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:01.498303 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631c9e3-8995-49b8-818b-392b6796ae21 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:01.515512 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631c9e3-8995-49b8-818b-392b6796ae21 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:01.521883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561fc18-5136-4831-ae0a-8440fd78ade8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:01.522479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561fc18-5136-4831-ae0a-8440fd78ade8 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:01.522784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561fc18-5136-4831-ae0a-8440fd78ade8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:01.523179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561fc18-5136-4831-ae0a-8440fd78ade8 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:01.524251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561fc18-5136-4831-ae0a-8440fd78ade8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:01.526773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561fc18-5136-4831-ae0a-8440fd78ade8 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:01.564195 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 817/1623] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:01 2022] POST /identity/v3/users => generated 289 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:01.568126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:01.569554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561fc18-5136-4831-ae0a-8440fd78ade8 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:01.584978 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 807/1624] 10.222.0.13 () {58 vars in 1058 bytes} [Sat Jun 25 16:06:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:01.586564 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74512c42-6b15-4232-927b-a617cf94f48b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:01.600921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561fc18-5136-4831-ae0a-8440fd78ade8 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:01.610995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74512c42-6b15-4232-927b-a617cf94f48b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:01.615076 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752c986-2fd4-4c93-bbd8-286a0a41361f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:01.616029 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752c986-2fd4-4c93-bbd8-286a0a41361f admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:01.616347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752c986-2fd4-4c93-bbd8-286a0a41361f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:01.616720 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2752c986-2fd4-4c93-bbd8-286a0a41361f admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:01.619263 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752c986-2fd4-4c93-bbd8-286a0a41361f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:01.619920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2752c986-2fd4-4c93-bbd8-286a0a41361f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:01.621452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d70d5f-ede7-448a-9c63-c991a0954122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:01.622288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d70d5f-ede7-448a-9c63-c991a0954122 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:01.622575 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d70d5f-ede7-448a-9c63-c991a0954122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:01.622984 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d70d5f-ede7-448a-9c63-c991a0954122 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:01.623690 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d70d5f-ede7-448a-9c63-c991a0954122 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:01.625744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d70d5f-ede7-448a-9c63-c991a0954122 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:01.653108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 808/1625] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:01 2022] POST /identity/v3/users => generated 309 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:01.657293 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d70d5f-ede7-448a-9c63-c991a0954122 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:01.669759 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d70d5f-ede7-448a-9c63-c991a0954122 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:01.676695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa7d7d-c862-49fb-b042-6407541c116c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:01.677862 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa7d7d-c862-49fb-b042-6407541c116c admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:01.678245 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa7d7d-c862-49fb-b042-6407541c116c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:01.678588 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa7d7d-c862-49fb-b042-6407541c116c admin admin] PATH_INFO: `/v3/projects/c1125e56256241b0bed82273007d693a/users/329294525a464b9c83070c6f17584bd3/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:01.689328 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 818/1626] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:01 2022] POST /identity/v3/projects => generated 405 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:01.692644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752c986-2fd4-4c93-bbd8-286a0a41361f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:01.700905 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:01.703201 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa7d7d-c862-49fb-b042-6407541c116c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1125e56256241b0bed82273007d693a, user_id=329294525a464b9c83070c6f17584bd3, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:01.707245 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa7d7d-c862-49fb-b042-6407541c116c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:01.737140 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 809/1627] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:01 2022] PUT /identity/v3/projects/c1125e56256241b0bed82273007d693a/users/329294525a464b9c83070c6f17584bd3/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:01.738366 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa7d7d-c862-49fb-b042-6407541c116c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:01.830084 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2752c986-2fd4-4c93-bbd8-286a0a41361f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:01.912639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa7d7d-c862-49fb-b042-6407541c116c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:01.928816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b30ed-c199-42e8-9fa1-89f3f72d4a83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:01.929511 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b30ed-c199-42e8-9fa1-89f3f72d4a83 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:01.929804 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b30ed-c199-42e8-9fa1-89f3f72d4a83 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:01.930586 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b30ed-c199-42e8-9fa1-89f3f72d4a83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:01.968353 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c87cd-fcfb-45d6-8b02-0c8b9bc41d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:01.970922 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c87cd-fcfb-45d6-8b02-0c8b9bc41d37 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:01.971287 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c87cd-fcfb-45d6-8b02-0c8b9bc41d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:01.971988 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c87cd-fcfb-45d6-8b02-0c8b9bc41d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:01.973963 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c87cd-fcfb-45d6-8b02-0c8b9bc41d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:01.976524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c87cd-fcfb-45d6-8b02-0c8b9bc41d37 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:02.001706 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 819/1628] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:02.002629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c87cd-fcfb-45d6-8b02-0c8b9bc41d37 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:02.023319 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c87cd-fcfb-45d6-8b02-0c8b9bc41d37 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:02.035035 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699f1d3-84ba-4ab6-a1da-6b0151ffde01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:02.035960 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699f1d3-84ba-4ab6-a1da-6b0151ffde01 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:02.036239 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699f1d3-84ba-4ab6-a1da-6b0151ffde01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:02.036631 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699f1d3-84ba-4ab6-a1da-6b0151ffde01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:02.037418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699f1d3-84ba-4ab6-a1da-6b0151ffde01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:02.039028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699f1d3-84ba-4ab6-a1da-6b0151ffde01 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:02.044573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b30ed-c199-42e8-9fa1-89f3f72d4a83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:02.046951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b30ed-c199-42e8-9fa1-89f3f72d4a83 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:02.053982 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 820/1629] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:02.058876 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699f1d3-84ba-4ab6-a1da-6b0151ffde01 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:02.072898 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 810/1630] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:02.075680 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b30ed-c199-42e8-9fa1-89f3f72d4a83 service nova] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:02.076879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699f1d3-84ba-4ab6-a1da-6b0151ffde01 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:02.082994 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c938783-e410-48ba-9284-91aaca884a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:02.083853 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c938783-e410-48ba-9284-91aaca884a2f admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:02.084169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c938783-e410-48ba-9284-91aaca884a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:02.084565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c938783-e410-48ba-9284-91aaca884a2f admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:02.085988 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c938783-e410-48ba-9284-91aaca884a2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:02.089730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c938783-e410-48ba-9284-91aaca884a2f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:02.106583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b30ed-c199-42e8-9fa1-89f3f72d4a83 service nova] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:02.121582 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708a8bb-7d9f-43f2-9c73-e7bfb2cdc354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:02.122853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708a8bb-7d9f-43f2-9c73-e7bfb2cdc354 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:02.123867 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708a8bb-7d9f-43f2-9c73-e7bfb2cdc354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:02.124654 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708a8bb-7d9f-43f2-9c73-e7bfb2cdc354 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:02.126641 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c708a8bb-7d9f-43f2-9c73-e7bfb2cdc354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:02.128850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 821/1631] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:02 2022] POST /identity/v3/users => generated 309 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:02.131388 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c708a8bb-7d9f-43f2-9c73-e7bfb2cdc354 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:02.132536 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c938783-e410-48ba-9284-91aaca884a2f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:02.163137 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 811/1632] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:02.164044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708a8bb-7d9f-43f2-9c73-e7bfb2cdc354 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:02.168017 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c938783-e410-48ba-9284-91aaca884a2f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:02.173148 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b03e2-6a35-4c94-8e7b-a2335ab085ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:02.174293 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b03e2-6a35-4c94-8e7b-a2335ab085ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:02.175169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b03e2-6a35-4c94-8e7b-a2335ab085ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:02.175562 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b03e2-6a35-4c94-8e7b-a2335ab085ed admin admin] PATH_INFO: `/v3/projects/e392401b028c4ef29f9959a543445c69/users/7fc1d0c6d5da470893e6e190b890369d/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:02.186253 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708a8bb-7d9f-43f2-9c73-e7bfb2cdc354 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:02.192110 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6245ec4-46b6-45c0-94ac-dd0a3bcbb4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:02.193260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6245ec4-46b6-45c0-94ac-dd0a3bcbb4ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:02.193545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6245ec4-46b6-45c0-94ac-dd0a3bcbb4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:02.193880 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6245ec4-46b6-45c0-94ac-dd0a3bcbb4ea admin admin] PATH_INFO: `/v3/projects/538138b3035d457ba0e7761a37251e6a/users/ed12bb2ebb184e68992bc1a08896b0d0/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:02.228157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b03e2-6a35-4c94-8e7b-a2335ab085ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=e392401b028c4ef29f9959a543445c69, user_id=7fc1d0c6d5da470893e6e190b890369d, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:02.237570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b03e2-6a35-4c94-8e7b-a2335ab085ed admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:02.241437 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6245ec4-46b6-45c0-94ac-dd0a3bcbb4ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=538138b3035d457ba0e7761a37251e6a, user_id=ed12bb2ebb184e68992bc1a08896b0d0, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:02.245224 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6245ec4-46b6-45c0-94ac-dd0a3bcbb4ea admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:02.271039 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 812/1633] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:02 2022] PUT /identity/v3/projects/538138b3035d457ba0e7761a37251e6a/users/ed12bb2ebb184e68992bc1a08896b0d0/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:02.271039 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6245ec4-46b6-45c0-94ac-dd0a3bcbb4ea admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:02.276591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 822/1634] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:02 2022] PUT /identity/v3/projects/e392401b028c4ef29f9959a543445c69/users/7fc1d0c6d5da470893e6e190b890369d/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:02.276591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b03e2-6a35-4c94-8e7b-a2335ab085ed admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:02.310154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:02.393977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6245ec4-46b6-45c0-94ac-dd0a3bcbb4ea admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:02.408438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:02.418856 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b03e2-6a35-4c94-8e7b-a2335ab085ed admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:02.436195 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40d967-30d7-480d-a02d-9a822df0e68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:02.437518 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40d967-30d7-480d-a02d-9a822df0e68f admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:02.438026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40d967-30d7-480d-a02d-9a822df0e68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:02.438478 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40d967-30d7-480d-a02d-9a822df0e68f admin admin] PATH_INFO: `/v3/projects/c1125e56256241b0bed82273007d693a/users/329294525a464b9c83070c6f17584bd3/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:02.443390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d40d967-30d7-480d-a02d-9a822df0e68f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1125e56256241b0bed82273007d693a, user_id=329294525a464b9c83070c6f17584bd3, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:02.448688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d40d967-30d7-480d-a02d-9a822df0e68f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:02.479751 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 823/1635] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:02 2022] PUT /identity/v3/projects/c1125e56256241b0bed82273007d693a/users/329294525a464b9c83070c6f17584bd3/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:02.491644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40d967-30d7-480d-a02d-9a822df0e68f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:02.546547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32172b89-49ef-42d0-a68a-d9e34dd2629d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:02.547300 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32172b89-49ef-42d0-a68a-d9e34dd2629d admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:02.547646 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32172b89-49ef-42d0-a68a-d9e34dd2629d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:02.547980 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32172b89-49ef-42d0-a68a-d9e34dd2629d admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:02.548873 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32172b89-49ef-42d0-a68a-d9e34dd2629d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:02.555745 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32172b89-49ef-42d0-a68a-d9e34dd2629d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:02.582568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 813/1636] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:02.582568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32172b89-49ef-42d0-a68a-d9e34dd2629d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:02.600070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32172b89-49ef-42d0-a68a-d9e34dd2629d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:02.630028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40d967-30d7-480d-a02d-9a822df0e68f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:02.652850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980740e8-c70e-4254-8ec2-cdd65a35f84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:02.652850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980740e8-c70e-4254-8ec2-cdd65a35f84e admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:02.652850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980740e8-c70e-4254-8ec2-cdd65a35f84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:02.653258 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980740e8-c70e-4254-8ec2-cdd65a35f84e admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:02.654103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980740e8-c70e-4254-8ec2-cdd65a35f84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:02.656077 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980740e8-c70e-4254-8ec2-cdd65a35f84e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:02.674383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 824/1637] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:02.675855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e95d62-4d2b-4acd-941d-21c9cd7982e6 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:02.676350 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e95d62-4d2b-4acd-941d-21c9cd7982e6 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:02.676719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e95d62-4d2b-4acd-941d-21c9cd7982e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:02.730133 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d2391-c970-4b8c-889b-356b6f53e627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:02.731033 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d2391-c970-4b8c-889b-356b6f53e627 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:02.731379 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d2391-c970-4b8c-889b-356b6f53e627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:02.731712 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d2391-c970-4b8c-889b-356b6f53e627 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:02.732999 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d2391-c970-4b8c-889b-356b6f53e627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:02.737436 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d2391-c970-4b8c-889b-356b6f53e627 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:02.739299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f4e95d62-4d2b-4acd-941d-21c9cd7982e6 None None] MFA Rules not processed for user `329294525a464b9c83070c6f17584bd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:02.741259 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f4e95d62-4d2b-4acd-941d-21c9cd7982e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:02.762909 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 814/1638] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:02.763857 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d2391-c970-4b8c-889b-356b6f53e627 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:02.789945 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d2391-c970-4b8c-889b-356b6f53e627 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:02.805105 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211be13c-5dbe-44f7-b405-7a5220f183a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:02.808519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211be13c-5dbe-44f7-b405-7a5220f183a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:02.808811 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211be13c-5dbe-44f7-b405-7a5220f183a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:02.809085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211be13c-5dbe-44f7-b405-7a5220f183a5 admin admin] PATH_INFO: `/v3/projects/a54ec262520c49838ffb02d54800597e/users/2a3a7c47864947edab213a6e69059650/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:02.834143 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211be13c-5dbe-44f7-b405-7a5220f183a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a54ec262520c49838ffb02d54800597e, user_id=2a3a7c47864947edab213a6e69059650, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:02.843732 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 825/1639] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:02 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:02.844725 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211be13c-5dbe-44f7-b405-7a5220f183a5 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:02.845446 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaeefcc-88a6-4362-aba6-015928433dd6 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:02.845852 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaeefcc-88a6-4362-aba6-015928433dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:02.846198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaeefcc-88a6-4362-aba6-015928433dd6 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:02.847937 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 826/1640] 10.222.0.13 () {54 vars in 912 bytes} [Sat Jun 25 16:06:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:06:02.848746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaeefcc-88a6-4362-aba6-015928433dd6 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:02.863072 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaeefcc-88a6-4362-aba6-015928433dd6 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:02.871783 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78897078-ea01-4443-ad69-4ebd56cdb062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:02.873650 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78897078-ea01-4443-ad69-4ebd56cdb062 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:02.873650 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78897078-ea01-4443-ad69-4ebd56cdb062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:02.873918 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78897078-ea01-4443-ad69-4ebd56cdb062 admin admin] PATH_INFO: `/v3/projects/538138b3035d457ba0e7761a37251e6a/users/ed12bb2ebb184e68992bc1a08896b0d0/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:02.877172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78897078-ea01-4443-ad69-4ebd56cdb062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=538138b3035d457ba0e7761a37251e6a, user_id=ed12bb2ebb184e68992bc1a08896b0d0, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:02.880596 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78897078-ea01-4443-ad69-4ebd56cdb062 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:02.884588 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 815/1641] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:02 2022] PUT /identity/v3/projects/a54ec262520c49838ffb02d54800597e/users/2a3a7c47864947edab213a6e69059650/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:02.888059 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211be13c-5dbe-44f7-b405-7a5220f183a5 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:02.904272 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 827/1642] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:02 2022] PUT /identity/v3/projects/538138b3035d457ba0e7761a37251e6a/users/ed12bb2ebb184e68992bc1a08896b0d0/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:02.904272 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd0a52-455c-4b13-80cc-a0f7a10da73b None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:02.904272 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd0a52-455c-4b13-80cc-a0f7a10da73b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:02.904272 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd0a52-455c-4b13-80cc-a0f7a10da73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:02.950386 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-d5bd0a52-455c-4b13-80cc-a0f7a10da73b None None] MFA Rules not processed for user `3622c5ccb7534e7c8e8342b4f6643bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:02.951782 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d5bd0a52-455c-4b13-80cc-a0f7a10da73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:02.975960 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:03.049541 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:03.050132 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211be13c-5dbe-44f7-b405-7a5220f183a5 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:03.076523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 828/1643] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Jun 25 16:06:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:03.077910 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd0a52-455c-4b13-80cc-a0f7a10da73b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:03.168812 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd0a52-455c-4b13-80cc-a0f7a10da73b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:03.176250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da2802-c44e-4767-ba0a-f718f912e531 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:03.176975 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da2802-c44e-4767-ba0a-f718f912e531 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:03.177263 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da2802-c44e-4767-ba0a-f718f912e531 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:03.178830 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da2802-c44e-4767-ba0a-f718f912e531 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:03.194150 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2d477-df87-440c-8e60-a88616a6df1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:03.195101 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2d477-df87-440c-8e60-a88616a6df1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:03.195387 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2d477-df87-440c-8e60-a88616a6df1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:03.195858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2d477-df87-440c-8e60-a88616a6df1f admin admin] PATH_INFO: `/v3/projects/e392401b028c4ef29f9959a543445c69/users/7fc1d0c6d5da470893e6e190b890369d/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:03.200038 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2d477-df87-440c-8e60-a88616a6df1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e392401b028c4ef29f9959a543445c69, user_id=7fc1d0c6d5da470893e6e190b890369d, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:03.204607 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2d477-df87-440c-8e60-a88616a6df1f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:03.239260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 816/1644] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:02 2022] PUT /identity/v3/projects/e392401b028c4ef29f9959a543445c69/users/7fc1d0c6d5da470893e6e190b890369d/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:03.240465 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2d477-df87-440c-8e60-a88616a6df1f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:03.312221 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da2802-c44e-4767-ba0a-f718f912e531 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:03.313149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da2802-c44e-4767-ba0a-f718f912e531 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:03.334355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 829/1645] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:03 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:03.335299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d158a8-bbe5-4f14-aedc-127b5efa23f8 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:03.335589 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d158a8-bbe5-4f14-aedc-127b5efa23f8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:03.335871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d158a8-bbe5-4f14-aedc-127b5efa23f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:03.341202 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2d477-df87-440c-8e60-a88616a6df1f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:03.347108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece2cfd-956b-4f78-b991-7d4f223fa63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:03.348974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece2cfd-956b-4f78-b991-7d4f223fa63d admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:03.350518 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece2cfd-956b-4f78-b991-7d4f223fa63d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:03.351011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece2cfd-956b-4f78-b991-7d4f223fa63d admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:03.352853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece2cfd-956b-4f78-b991-7d4f223fa63d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:03.354853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece2cfd-956b-4f78-b991-7d4f223fa63d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:03.368995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 817/1646] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:03.370721 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece2cfd-956b-4f78-b991-7d4f223fa63d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:03.394156 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b0d158a8-bbe5-4f14-aedc-127b5efa23f8 None None] MFA Rules not processed for user `ed12bb2ebb184e68992bc1a08896b0d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:03.395411 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b0d158a8-bbe5-4f14-aedc-127b5efa23f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:03.525843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece2cfd-956b-4f78-b991-7d4f223fa63d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:03.526658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 830/1647] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:03 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:03.528074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fdcc2-af8c-4fb2-9af1-7f8f2648b9a4 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:03.529135 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fdcc2-af8c-4fb2-9af1-7f8f2648b9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:03.529517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fdcc2-af8c-4fb2-9af1-7f8f2648b9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:03.529990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9f194-3b56-4e98-bd73-9724e42df859 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:03.530651 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9f194-3b56-4e98-bd73-9724e42df859 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:03.530927 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9f194-3b56-4e98-bd73-9724e42df859 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:03.531274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9f194-3b56-4e98-bd73-9724e42df859 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:03.578701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:03.604604 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3d1fdcc2-af8c-4fb2-9af1-7f8f2648b9a4 None None] MFA Rules not processed for user `7fc1d0c6d5da470893e6e190b890369d`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:03.607367 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3d1fdcc2-af8c-4fb2-9af1-7f8f2648b9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:03.626127 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:03.707690 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9f194-3b56-4e98-bd73-9724e42df859 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:03.711514 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9f194-3b56-4e98-bd73-9724e42df859 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:03.753589 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 831/1648] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:03 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:03.754545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 818/1649] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:03.755600 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9f194-3b56-4e98-bd73-9724e42df859 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:03.756837 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1fdcc2-af8c-4fb2-9af1-7f8f2648b9a4 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:03.779093 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9f194-3b56-4e98-bd73-9724e42df859 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:03.795447 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1fdcc2-af8c-4fb2-9af1-7f8f2648b9a4 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:03.796057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e85efa-fae6-49b6-9fd4-da8585565073 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:03.796922 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e85efa-fae6-49b6-9fd4-da8585565073 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:03.797310 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e85efa-fae6-49b6-9fd4-da8585565073 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:03.797662 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e85efa-fae6-49b6-9fd4-da8585565073 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:03.814114 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746fabb-f58d-4ddf-ac27-2447c0dfedc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:03.815066 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746fabb-f58d-4ddf-ac27-2447c0dfedc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:03.815816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746fabb-f58d-4ddf-ac27-2447c0dfedc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:03.816736 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746fabb-f58d-4ddf-ac27-2447c0dfedc5 admin admin] PATH_INFO: `/v3/projects/a54ec262520c49838ffb02d54800597e/users/2a3a7c47864947edab213a6e69059650/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:03.818306 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f746fabb-f58d-4ddf-ac27-2447c0dfedc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a54ec262520c49838ffb02d54800597e, user_id=2a3a7c47864947edab213a6e69059650, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:03.821002 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f746fabb-f58d-4ddf-ac27-2447c0dfedc5 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:03.824446 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e85efa-fae6-49b6-9fd4-da8585565073 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:03.826649 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e85efa-fae6-49b6-9fd4-da8585565073 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:03.869159 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 832/1650] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:03 2022] PUT /identity/v3/projects/a54ec262520c49838ffb02d54800597e/users/2a3a7c47864947edab213a6e69059650/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:03.869346 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 819/1651] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:03 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:04.065312 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746fabb-f58d-4ddf-ac27-2447c0dfedc5 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:04.171425 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746fabb-f58d-4ddf-ac27-2447c0dfedc5 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:04.182761 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d92d40-5916-4a21-9121-45b4f3029b9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:04.183457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d92d40-5916-4a21-9121-45b4f3029b9a service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:04.183741 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d92d40-5916-4a21-9121-45b4f3029b9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:04.184020 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d92d40-5916-4a21-9121-45b4f3029b9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:04.268294 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d92d40-5916-4a21-9121-45b4f3029b9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:04.269978 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d92d40-5916-4a21-9121-45b4f3029b9a service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:04.283125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 833/1652] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:04 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:04.700045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e85efa-fae6-49b6-9fd4-da8585565073 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:04.710824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e85efa-fae6-49b6-9fd4-da8585565073 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:04.720845 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f8702-0f88-40af-8db6-961132f09a94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:04.721512 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f8702-0f88-40af-8db6-961132f09a94 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:04.721790 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f8702-0f88-40af-8db6-961132f09a94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:04.722058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f8702-0f88-40af-8db6-961132f09a94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:04.871311 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d92d40-5916-4a21-9121-45b4f3029b9a service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:04.887407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f8702-0f88-40af-8db6-961132f09a94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:04.892542 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f8702-0f88-40af-8db6-961132f09a94 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:04.926352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 820/1653] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:04 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:05.150931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:05.216529 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d92d40-5916-4a21-9121-45b4f3029b9a service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:05.246097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dcff0b-43d8-4e20-a377-4eb160b15f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:05.247044 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcff0b-43d8-4e20-a377-4eb160b15f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:05.247371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcff0b-43d8-4e20-a377-4eb160b15f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:05.247650 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcff0b-43d8-4e20-a377-4eb160b15f93 admin admin] PATH_INFO: `/v3/users/a069e0ece7974f0b9072be533518980a` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:05.249835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dcff0b-43d8-4e20-a377-4eb160b15f93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a069e0ece7974f0b9072be533518980a)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:05.252587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dcff0b-43d8-4e20-a377-4eb160b15f93 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:05.420319 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-36dcff0b-43d8-4e20-a377-4eb160b15f93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a069e0ece7974f0b9072be533518980a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:05.453125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-36dcff0b-43d8-4e20-a377-4eb160b15f93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a069e0ece7974f0b9072be533518980a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:05.455441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-36dcff0b-43d8-4e20-a377-4eb160b15f93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a069e0ece7974f0b9072be533518980a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:05.476254 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-36dcff0b-43d8-4e20-a377-4eb160b15f93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a069e0ece7974f0b9072be533518980a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:05.477907 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-36dcff0b-43d8-4e20-a377-4eb160b15f93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a069e0ece7974f0b9072be533518980a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:05.595711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737137aa-a715-4d45-b332-1bebeb8ed7fd None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:05.595711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737137aa-a715-4d45-b332-1bebeb8ed7fd None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:05.595711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737137aa-a715-4d45-b332-1bebeb8ed7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:05.599456 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 834/1654] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:04 2022] DELETE /identity/v3/users/a069e0ece7974f0b9072be533518980a => generated 0 bytes in 720 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:06:05.647382 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-737137aa-a715-4d45-b332-1bebeb8ed7fd None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:05.655261 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-737137aa-a715-4d45-b332-1bebeb8ed7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:05.871650 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 821/1655] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:05.940597 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dcff0b-43d8-4e20-a377-4eb160b15f93 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:05.950955 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-36dcff0b-43d8-4e20-a377-4eb160b15f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:06.006083 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737137aa-a715-4d45-b332-1bebeb8ed7fd None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:06.007919 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-737137aa-a715-4d45-b332-1bebeb8ed7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:06.028628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737137aa-a715-4d45-b332-1bebeb8ed7fd None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:06.034447 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e0ec4-d55b-4b07-96c9-6435d50bf301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:06.035709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e0ec4-d55b-4b07-96c9-6435d50bf301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:06.036889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e0ec4-d55b-4b07-96c9-6435d50bf301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:06.037378 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e0ec4-d55b-4b07-96c9-6435d50bf301 admin admin] PATH_INFO: `/v3/users/755c0268c22a479d983dba06c4122a90` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:06.040064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e0ec4-d55b-4b07-96c9-6435d50bf301 admin admin] RBAC: Authorizing `identity:delete_user(user_id=755c0268c22a479d983dba06c4122a90)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:06.043714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: WARNING py.warnings [None req-d52e0ec4-d55b-4b07-96c9-6435d50bf301 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 25 16:06:06.043714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: warnings.warn(msg) Jun 25 16:06:06.043714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: Jun 25 16:06:06.045611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e0ec4-d55b-4b07-96c9-6435d50bf301 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:06.074435 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dcff0b-43d8-4e20-a377-4eb160b15f93 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:06.078313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87f43a-c069-4234-a738-c27cddba9645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:06.078987 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87f43a-c069-4234-a738-c27cddba9645 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:06.079263 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87f43a-c069-4234-a738-c27cddba9645 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:06.079536 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87f43a-c069-4234-a738-c27cddba9645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:06.094936 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87f43a-c069-4234-a738-c27cddba9645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:06.096386 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87f43a-c069-4234-a738-c27cddba9645 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:06.125281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 835/1656] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:06.155896 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:06.184132 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d52e0ec4-d55b-4b07-96c9-6435d50bf301 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '755c0268c22a479d983dba06c4122a90'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:06.226172 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d52e0ec4-d55b-4b07-96c9-6435d50bf301 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '755c0268c22a479d983dba06c4122a90'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:06.226172 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d52e0ec4-d55b-4b07-96c9-6435d50bf301 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '755c0268c22a479d983dba06c4122a90'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:06.227865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065108f-6e51-4ddc-b2ed-ca6244b9bbe9 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:06.227865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065108f-6e51-4ddc-b2ed-ca6244b9bbe9 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:06.227865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065108f-6e51-4ddc-b2ed-ca6244b9bbe9 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:06.227865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 836/1657] 10.222.0.13 () {54 vars in 961 bytes} [Sat Jun 25 16:06:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:06:06.241996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d61018-3838-4138-90f8-e82a8ce8e986 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:06.242579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d61018-3838-4138-90f8-e82a8ce8e986 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:06.243460 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d61018-3838-4138-90f8-e82a8ce8e986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:06.244237 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d52e0ec4-d55b-4b07-96c9-6435d50bf301 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '755c0268c22a479d983dba06c4122a90'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:06.249106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d52e0ec4-d55b-4b07-96c9-6435d50bf301 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '755c0268c22a479d983dba06c4122a90'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:06.298909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-41d61018-3838-4138-90f8-e82a8ce8e986 None None] MFA Rules not processed for user `3622c5ccb7534e7c8e8342b4f6643bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:06.300157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-41d61018-3838-4138-90f8-e82a8ce8e986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:06.330392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 822/1658] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:06 2022] DELETE /identity/v3/users/755c0268c22a479d983dba06c4122a90 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:06.333725 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a591c-43dc-4308-b695-b48588fc3093 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:06.334246 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a591c-43dc-4308-b695-b48588fc3093 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:06.334523 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a591c-43dc-4308-b695-b48588fc3093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:06.373296 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ad1a591c-43dc-4308-b695-b48588fc3093 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:06.379948 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ad1a591c-43dc-4308-b695-b48588fc3093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:06.441580 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:06.470787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 837/1659] 10.222.0.13 () {58 vars in 1058 bytes} [Sat Jun 25 16:06:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:06.472361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf85a7e-4c02-43f1-bc28-95c284b7b4c4 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:06.479759 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf85a7e-4c02-43f1-bc28-95c284b7b4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:06.480062 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf85a7e-4c02-43f1-bc28-95c284b7b4c4 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:06.486538 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 838/1660] 10.222.0.13 () {54 vars in 912 bytes} [Sat Jun 25 16:06:06 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:06:06.488163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a62c10-d184-41bb-a16c-85e8be05089a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:06.488493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a62c10-d184-41bb-a16c-85e8be05089a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:06.488783 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a62c10-d184-41bb-a16c-85e8be05089a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:06.533237 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-55a62c10-d184-41bb-a16c-85e8be05089a None None] MFA Rules not processed for user `3622c5ccb7534e7c8e8342b4f6643bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:06.537732 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-55a62c10-d184-41bb-a16c-85e8be05089a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:06.541481 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 823/1661] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:06.589393 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a591c-43dc-4308-b695-b48588fc3093 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:06.591946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ad1a591c-43dc-4308-b695-b48588fc3093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:06.759426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 839/1662] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Jun 25 16:06:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:06.776918 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a62c10-d184-41bb-a16c-85e8be05089a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:06.778917 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-55a62c10-d184-41bb-a16c-85e8be05089a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:06.846025 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a591c-43dc-4308-b695-b48588fc3093 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:06.862261 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc691136-1f66-4ae7-80a3-0675f228a2c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:06.862887 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc691136-1f66-4ae7-80a3-0675f228a2c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:06.863171 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc691136-1f66-4ae7-80a3-0675f228a2c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:06.867681 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc691136-1f66-4ae7-80a3-0675f228a2c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:06.908634 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc691136-1f66-4ae7-80a3-0675f228a2c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:06.919188 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc691136-1f66-4ae7-80a3-0675f228a2c9 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:06.973803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 824/1663] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:06.997206 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a62c10-d184-41bb-a16c-85e8be05089a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:07.001593 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3790cab-1008-4924-92aa-f8749820c18f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:07.002348 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3790cab-1008-4924-92aa-f8749820c18f service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:07.002678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3790cab-1008-4924-92aa-f8749820c18f service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:07.003006 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3790cab-1008-4924-92aa-f8749820c18f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:07.017316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3790cab-1008-4924-92aa-f8749820c18f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:07.019736 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3790cab-1008-4924-92aa-f8749820c18f service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:07.081178 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 840/1664] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:07.171988 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50942a67-1c60-4b55-a9e0-4272dab93d06 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:07.172362 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50942a67-1c60-4b55-a9e0-4272dab93d06 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:07.172851 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50942a67-1c60-4b55-a9e0-4272dab93d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:07.225432 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-50942a67-1c60-4b55-a9e0-4272dab93d06 None None] MFA Rules not processed for user `2a3a7c47864947edab213a6e69059650`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:07.229065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-50942a67-1c60-4b55-a9e0-4272dab93d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:07.353335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:07.428632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 825/1665] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:07 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:06:08.890784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3790cab-1008-4924-92aa-f8749820c18f service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:08.892851 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e3790cab-1008-4924-92aa-f8749820c18f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:08.908386 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3790cab-1008-4924-92aa-f8749820c18f service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:08.913382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046adcc-28e2-497a-8ee2-c4ed8fdc4b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:08.914687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046adcc-28e2-497a-8ee2-c4ed8fdc4b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:08.915135 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046adcc-28e2-497a-8ee2-c4ed8fdc4b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:08.915546 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046adcc-28e2-497a-8ee2-c4ed8fdc4b34 admin admin] PATH_INFO: `/v3/projects/0b4842ea7c7a463598d269c34e16c7ba` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:08.916937 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046adcc-28e2-497a-8ee2-c4ed8fdc4b34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b4842ea7c7a463598d269c34e16c7ba)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:08.918086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: WARNING py.warnings [None req-f046adcc-28e2-497a-8ee2-c4ed8fdc4b34 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 25 16:06:08.918086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: warnings.warn(msg) Jun 25 16:06:08.918086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: Jun 25 16:06:08.919879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046adcc-28e2-497a-8ee2-c4ed8fdc4b34 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:09.016442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f046adcc-28e2-497a-8ee2-c4ed8fdc4b34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b4842ea7c7a463598d269c34e16c7ba', 'request_id': 'req-f046adcc-28e2-497a-8ee2-c4ed8fdc4b34'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:09.028598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f046adcc-28e2-497a-8ee2-c4ed8fdc4b34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b4842ea7c7a463598d269c34e16c7ba', 'request_id': 'req-f046adcc-28e2-497a-8ee2-c4ed8fdc4b34'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:09.052266 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f046adcc-28e2-497a-8ee2-c4ed8fdc4b34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b4842ea7c7a463598d269c34e16c7ba', 'request_id': 'req-f046adcc-28e2-497a-8ee2-c4ed8fdc4b34'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:09.092582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f046adcc-28e2-497a-8ee2-c4ed8fdc4b34 admin admin] The token cache is being invalidate because project 0b4842ea7c7a463598d269c34e16c7ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:09.093025 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f046adcc-28e2-497a-8ee2-c4ed8fdc4b34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:09.094254 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f046adcc-28e2-497a-8ee2-c4ed8fdc4b34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:09.098759 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 841/1666] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:08 2022] DELETE /identity/v3/projects/0b4842ea7c7a463598d269c34e16c7ba => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:09.100265 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50942a67-1c60-4b55-a9e0-4272dab93d06 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:09.126053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50942a67-1c60-4b55-a9e0-4272dab93d06 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:09.205500 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bcd19-d295-4c31-9b0d-7e6c0de225e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:09.206942 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bcd19-d295-4c31-9b0d-7e6c0de225e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:09.207467 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bcd19-d295-4c31-9b0d-7e6c0de225e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:09.207925 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bcd19-d295-4c31-9b0d-7e6c0de225e6 admin admin] PATH_INFO: `/v3/users/ed12bb2ebb184e68992bc1a08896b0d0` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:09.210290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5bcd19-d295-4c31-9b0d-7e6c0de225e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed12bb2ebb184e68992bc1a08896b0d0)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:09.212645 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5bcd19-d295-4c31-9b0d-7e6c0de225e6 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:09.314944 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1f5bcd19-d295-4c31-9b0d-7e6c0de225e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed12bb2ebb184e68992bc1a08896b0d0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:09.316130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1f5bcd19-d295-4c31-9b0d-7e6c0de225e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed12bb2ebb184e68992bc1a08896b0d0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:09.330900 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1f5bcd19-d295-4c31-9b0d-7e6c0de225e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed12bb2ebb184e68992bc1a08896b0d0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:09.343744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1f5bcd19-d295-4c31-9b0d-7e6c0de225e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed12bb2ebb184e68992bc1a08896b0d0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:09.345581 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1f5bcd19-d295-4c31-9b0d-7e6c0de225e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed12bb2ebb184e68992bc1a08896b0d0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:09.354016 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f3f8f-ce70-45d6-a616-9dfbac40707c None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:09.354702 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f3f8f-ce70-45d6-a616-9dfbac40707c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:09.355168 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f3f8f-ce70-45d6-a616-9dfbac40707c None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:09.357180 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 842/1667] 10.222.0.13 () {54 vars in 921 bytes} [Sat Jun 25 16:06:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:06:09.381168 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d341826-03cf-4462-b9a8-7a762542a2c0 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:09.381755 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d341826-03cf-4462-b9a8-7a762542a2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:09.382323 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d341826-03cf-4462-b9a8-7a762542a2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:09.393728 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 826/1668] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:09 2022] DELETE /identity/v3/users/ed12bb2ebb184e68992bc1a08896b0d0 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:09.395339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bcd19-d295-4c31-9b0d-7e6c0de225e6 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:09.398149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1f5bcd19-d295-4c31-9b0d-7e6c0de225e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:09.426320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3d341826-03cf-4462-b9a8-7a762542a2c0 None None] MFA Rules not processed for user `474c468a610140f8a5b18d5a1ed1d18e`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:09.429267 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3d341826-03cf-4462-b9a8-7a762542a2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:09.460672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:09.520028 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:09.548588 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bcd19-d295-4c31-9b0d-7e6c0de225e6 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:09.560203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 843/1669] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Jun 25 16:06:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:06:09.562579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aee4f0-cf40-4698-b918-72b5b6b9cfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:09.563653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aee4f0-cf40-4698-b918-72b5b6b9cfa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:09.563753 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aee4f0-cf40-4698-b918-72b5b6b9cfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:09.564014 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aee4f0-cf40-4698-b918-72b5b6b9cfa5 admin admin] PATH_INFO: `/v3/projects/3afb1695c99040d9a9812815a5e7b419` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:09.567520 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aee4f0-cf40-4698-b918-72b5b6b9cfa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3afb1695c99040d9a9812815a5e7b419)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:09.569568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aee4f0-cf40-4698-b918-72b5b6b9cfa5 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:09.595771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d341826-03cf-4462-b9a8-7a762542a2c0 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:09.598056 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3d341826-03cf-4462-b9a8-7a762542a2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:09.616000 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d341826-03cf-4462-b9a8-7a762542a2c0 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:09.624840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d1360-523d-4f5a-a7a4-e2e4c4fdb9a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474c468a610140f8a5b18d5a1ed1d18e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:09.626044 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d1360-523d-4f5a-a7a4-e2e4c4fdb9a5 service placement] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:09.626755 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d1360-523d-4f5a-a7a4-e2e4c4fdb9a5 service placement] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:09.627478 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d1360-523d-4f5a-a7a4-e2e4c4fdb9a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:09.646389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d1360-523d-4f5a-a7a4-e2e4c4fdb9a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:09.649873 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d1360-523d-4f5a-a7a4-e2e4c4fdb9a5 service placement] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:09.668313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 844/1670] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:09.669524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d1360-523d-4f5a-a7a4-e2e4c4fdb9a5 service placement] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:09.671507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c46d1360-523d-4f5a-a7a4-e2e4c4fdb9a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:09.679285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-45aee4f0-cf40-4698-b918-72b5b6b9cfa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3afb1695c99040d9a9812815a5e7b419', 'request_id': 'req-45aee4f0-cf40-4698-b918-72b5b6b9cfa5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:09.688441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d1360-523d-4f5a-a7a4-e2e4c4fdb9a5 service placement] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:09.690283 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-45aee4f0-cf40-4698-b918-72b5b6b9cfa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3afb1695c99040d9a9812815a5e7b419', 'request_id': 'req-45aee4f0-cf40-4698-b918-72b5b6b9cfa5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:09.699834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-45aee4f0-cf40-4698-b918-72b5b6b9cfa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3afb1695c99040d9a9812815a5e7b419', 'request_id': 'req-45aee4f0-cf40-4698-b918-72b5b6b9cfa5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:09.722092 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-45aee4f0-cf40-4698-b918-72b5b6b9cfa5 admin admin] The token cache is being invalidate because project 3afb1695c99040d9a9812815a5e7b419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:09.722411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-45aee4f0-cf40-4698-b918-72b5b6b9cfa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:09.723111 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-45aee4f0-cf40-4698-b918-72b5b6b9cfa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:09.724956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 827/1671] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:09 2022] DELETE /identity/v3/projects/3afb1695c99040d9a9812815a5e7b419 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:09.729827 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aee4f0-cf40-4698-b918-72b5b6b9cfa5 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:09.732717 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-45aee4f0-cf40-4698-b918-72b5b6b9cfa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:09.812153 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c74035-e263-4f95-a17a-ff0c61a6728b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:09.813108 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c74035-e263-4f95-a17a-ff0c61a6728b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:09.813388 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c74035-e263-4f95-a17a-ff0c61a6728b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:09.813733 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c74035-e263-4f95-a17a-ff0c61a6728b admin admin] PATH_INFO: `/v3/users/def03b41c32941778ab452bd83982949` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:09.816267 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c74035-e263-4f95-a17a-ff0c61a6728b admin admin] RBAC: Authorizing `identity:delete_user(user_id=def03b41c32941778ab452bd83982949)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:09.820002 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c74035-e263-4f95-a17a-ff0c61a6728b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:09.940892 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aee4f0-cf40-4698-b918-72b5b6b9cfa5 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:09.969936 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba812a11-b64a-4719-8e6c-cac179df6b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:09.970535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-22c74035-e263-4f95-a17a-ff0c61a6728b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'def03b41c32941778ab452bd83982949'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:09.971740 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba812a11-b64a-4719-8e6c-cac179df6b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:09.972159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba812a11-b64a-4719-8e6c-cac179df6b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:09.972633 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba812a11-b64a-4719-8e6c-cac179df6b6d admin admin] PATH_INFO: `/v3/users/329294525a464b9c83070c6f17584bd3` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:09.974036 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba812a11-b64a-4719-8e6c-cac179df6b6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=329294525a464b9c83070c6f17584bd3)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:09.976593 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba812a11-b64a-4719-8e6c-cac179df6b6d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:09.985975 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-22c74035-e263-4f95-a17a-ff0c61a6728b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'def03b41c32941778ab452bd83982949'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:09.987592 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-22c74035-e263-4f95-a17a-ff0c61a6728b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'def03b41c32941778ab452bd83982949'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:10.009553 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-22c74035-e263-4f95-a17a-ff0c61a6728b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'def03b41c32941778ab452bd83982949'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:10.011016 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-22c74035-e263-4f95-a17a-ff0c61a6728b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'def03b41c32941778ab452bd83982949'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:10.052659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 845/1672] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:09 2022] DELETE /identity/v3/users/def03b41c32941778ab452bd83982949 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:10.058227 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bc7ba-eabd-4282-80fd-a3c7b3021ab5 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:10.059035 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bc7ba-eabd-4282-80fd-a3c7b3021ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:10.059035 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bc7ba-eabd-4282-80fd-a3c7b3021ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:10.108300 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-fb2bc7ba-eabd-4282-80fd-a3c7b3021ab5 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:10.113839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-fb2bc7ba-eabd-4282-80fd-a3c7b3021ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:10.122082 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ba812a11-b64a-4719-8e6c-cac179df6b6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '329294525a464b9c83070c6f17584bd3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:10.123751 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ba812a11-b64a-4719-8e6c-cac179df6b6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '329294525a464b9c83070c6f17584bd3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:10.140869 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ba812a11-b64a-4719-8e6c-cac179df6b6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '329294525a464b9c83070c6f17584bd3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:10.150605 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:10.159443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ba812a11-b64a-4719-8e6c-cac179df6b6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '329294525a464b9c83070c6f17584bd3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:10.161582 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ba812a11-b64a-4719-8e6c-cac179df6b6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '329294525a464b9c83070c6f17584bd3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:10.203843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 828/1673] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:09 2022] DELETE /identity/v3/users/329294525a464b9c83070c6f17584bd3 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:06:10.255877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 846/1674] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:10.271739 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba812a11-b64a-4719-8e6c-cac179df6b6d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:10.274647 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ba812a11-b64a-4719-8e6c-cac179df6b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:10.283034 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:10.448238 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba812a11-b64a-4719-8e6c-cac179df6b6d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:10.461640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8696c17-2b38-49d3-8cda-7b2baef80fff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:10.462758 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8696c17-2b38-49d3-8cda-7b2baef80fff service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:10.462859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8696c17-2b38-49d3-8cda-7b2baef80fff service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:10.463128 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8696c17-2b38-49d3-8cda-7b2baef80fff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:10.630953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8696c17-2b38-49d3-8cda-7b2baef80fff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:10.632568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8696c17-2b38-49d3-8cda-7b2baef80fff service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:10.651130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 829/1675] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:10.842559 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a6bf8-7caa-4d08-84c6-810b937a778e None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:10.842852 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a6bf8-7caa-4d08-84c6-810b937a778e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:10.843132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a6bf8-7caa-4d08-84c6-810b937a778e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:10.913892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e16a6bf8-7caa-4d08-84c6-810b937a778e None None] MFA Rules not processed for user `8a3137566b004edb909d4e3c702a7f59`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:10.916301 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e16a6bf8-7caa-4d08-84c6-810b937a778e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:11.055999 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 847/1676] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:11.058105 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26180652-35ae-40fc-9bad-b9bbed19ca61 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:11.058259 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26180652-35ae-40fc-9bad-b9bbed19ca61 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:11.058538 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26180652-35ae-40fc-9bad-b9bbed19ca61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:11.093772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-26180652-35ae-40fc-9bad-b9bbed19ca61 None None] MFA Rules not processed for user `8a3137566b004edb909d4e3c702a7f59`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:11.095837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-26180652-35ae-40fc-9bad-b9bbed19ca61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:11.107449 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 830/1677] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:11.114887 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a6bf8-7caa-4d08-84c6-810b937a778e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:11.116751 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e16a6bf8-7caa-4d08-84c6-810b937a778e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:11.284024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a6bf8-7caa-4d08-84c6-810b937a778e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:11.301710 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936c61f-876c-4438-b6f4-111feafff6e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:11.302518 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936c61f-876c-4438-b6f4-111feafff6e0 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:11.302866 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936c61f-876c-4438-b6f4-111feafff6e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:11.303369 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936c61f-876c-4438-b6f4-111feafff6e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:11.333924 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b936c61f-876c-4438-b6f4-111feafff6e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:11.335551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b936c61f-876c-4438-b6f4-111feafff6e0 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:11.345991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26180652-35ae-40fc-9bad-b9bbed19ca61 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:11.347205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 848/1678] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:11.348082 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-26180652-35ae-40fc-9bad-b9bbed19ca61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:11.379099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26180652-35ae-40fc-9bad-b9bbed19ca61 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:11.385772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274c949-6799-472b-b3e8-88a750f1a5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:11.386785 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274c949-6799-472b-b3e8-88a750f1a5b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:11.387070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274c949-6799-472b-b3e8-88a750f1a5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:11.387343 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274c949-6799-472b-b3e8-88a750f1a5b7 admin admin] PATH_INFO: `/v3/projects/538138b3035d457ba0e7761a37251e6a` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:11.389119 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274c949-6799-472b-b3e8-88a750f1a5b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=538138b3035d457ba0e7761a37251e6a)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:11.394150 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274c949-6799-472b-b3e8-88a750f1a5b7 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:11.509671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a274c949-6799-472b-b3e8-88a750f1a5b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '538138b3035d457ba0e7761a37251e6a', 'request_id': 'req-a274c949-6799-472b-b3e8-88a750f1a5b7'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:11.521799 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a274c949-6799-472b-b3e8-88a750f1a5b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '538138b3035d457ba0e7761a37251e6a', 'request_id': 'req-a274c949-6799-472b-b3e8-88a750f1a5b7'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:11.537610 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a274c949-6799-472b-b3e8-88a750f1a5b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '538138b3035d457ba0e7761a37251e6a', 'request_id': 'req-a274c949-6799-472b-b3e8-88a750f1a5b7'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:11.569727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a274c949-6799-472b-b3e8-88a750f1a5b7 admin admin] The token cache is being invalidate because project 538138b3035d457ba0e7761a37251e6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:11.569943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a274c949-6799-472b-b3e8-88a750f1a5b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:11.571648 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a274c949-6799-472b-b3e8-88a750f1a5b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:11.574081 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 831/1679] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:11 2022] DELETE /identity/v3/projects/538138b3035d457ba0e7761a37251e6a => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:11.578953 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a61e5-2c59-4f3e-9fac-f5a3b88b15b7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:11.579336 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a61e5-2c59-4f3e-9fac-f5a3b88b15b7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:11.580041 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a61e5-2c59-4f3e-9fac-f5a3b88b15b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:11.614585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c79a61e5-2c59-4f3e-9fac-f5a3b88b15b7 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:11.615016 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c79a61e5-2c59-4f3e-9fac-f5a3b88b15b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:11.728265 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:11.785238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 849/1680] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:11.793048 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2698a34a-93f5-4d7e-b231-7c21d74efc5f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:11.793359 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2698a34a-93f5-4d7e-b231-7c21d74efc5f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:11.793608 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2698a34a-93f5-4d7e-b231-7c21d74efc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:11.813886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a61e5-2c59-4f3e-9fac-f5a3b88b15b7 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:11.817641 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c79a61e5-2c59-4f3e-9fac-f5a3b88b15b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:11.851526 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2698a34a-93f5-4d7e-b231-7c21d74efc5f None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:11.854827 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2698a34a-93f5-4d7e-b231-7c21d74efc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:11.878247 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a61e5-2c59-4f3e-9fac-f5a3b88b15b7 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:11.893435 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 832/1681] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:11.894105 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512060a-19aa-4578-8818-ca1628ba3bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:11.895044 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512060a-19aa-4578-8818-ca1628ba3bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:11.895288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512060a-19aa-4578-8818-ca1628ba3bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:11.895570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512060a-19aa-4578-8818-ca1628ba3bd2 admin admin] PATH_INFO: `/v3/projects/c1125e56256241b0bed82273007d693a` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:11.897810 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2698a34a-93f5-4d7e-b231-7c21d74efc5f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:11.899985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512060a-19aa-4578-8818-ca1628ba3bd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1125e56256241b0bed82273007d693a)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:11.905327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512060a-19aa-4578-8818-ca1628ba3bd2 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:11.928070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2698a34a-93f5-4d7e-b231-7c21d74efc5f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:11.945974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44014773-3223-4477-9592-443cd0cd8271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:11.947756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44014773-3223-4477-9592-443cd0cd8271 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:11.947756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44014773-3223-4477-9592-443cd0cd8271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:11.947756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44014773-3223-4477-9592-443cd0cd8271 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:11.948840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44014773-3223-4477-9592-443cd0cd8271 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:06:11.948840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44014773-3223-4477-9592-443cd0cd8271 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:11.951007 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44014773-3223-4477-9592-443cd0cd8271 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:12.017446 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 833/1682] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:06:12.018348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44014773-3223-4477-9592-443cd0cd8271 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:12.022688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6512060a-19aa-4578-8818-ca1628ba3bd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1125e56256241b0bed82273007d693a', 'request_id': 'req-6512060a-19aa-4578-8818-ca1628ba3bd2'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:12.037563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6512060a-19aa-4578-8818-ca1628ba3bd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1125e56256241b0bed82273007d693a', 'request_id': 'req-6512060a-19aa-4578-8818-ca1628ba3bd2'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:12.045526 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44014773-3223-4477-9592-443cd0cd8271 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:12.051058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6512060a-19aa-4578-8818-ca1628ba3bd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1125e56256241b0bed82273007d693a', 'request_id': 'req-6512060a-19aa-4578-8818-ca1628ba3bd2'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:12.085706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:12.108927 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6512060a-19aa-4578-8818-ca1628ba3bd2 admin admin] The token cache is being invalidate because project c1125e56256241b0bed82273007d693a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:12.116979 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6512060a-19aa-4578-8818-ca1628ba3bd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:12.120766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6512060a-19aa-4578-8818-ca1628ba3bd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:12.123893 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 850/1683] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:11 2022] DELETE /identity/v3/projects/c1125e56256241b0bed82273007d693a => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:06:12.133322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2cab-ad81-4ee8-b9a1-6776d78ba18c None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:12.135453 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2cab-ad81-4ee8-b9a1-6776d78ba18c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:12.137697 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2cab-ad81-4ee8-b9a1-6776d78ba18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:12.190904 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b58f2cab-ad81-4ee8-b9a1-6776d78ba18c None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:12.194412 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b58f2cab-ad81-4ee8-b9a1-6776d78ba18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:12.259424 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ef2e3-5983-47e0-8216-28ad1eed29d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:12.260263 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ef2e3-5983-47e0-8216-28ad1eed29d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:12.260578 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ef2e3-5983-47e0-8216-28ad1eed29d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:12.260855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ef2e3-5983-47e0-8216-28ad1eed29d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:12.262278 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ef2e3-5983-47e0-8216-28ad1eed29d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:12.269051 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ef2e3-5983-47e0-8216-28ad1eed29d1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:12.336537 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 834/1684] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:12 2022] POST /identity/v3/projects => generated 369 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:12.338052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ef2e3-5983-47e0-8216-28ad1eed29d1 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:12.340853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-697ef2e3-5983-47e0-8216-28ad1eed29d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:12.366132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 851/1685] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:12.367133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f2cab-ad81-4ee8-b9a1-6776d78ba18c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:12.372366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b58f2cab-ad81-4ee8-b9a1-6776d78ba18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:12.399152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f2cab-ad81-4ee8-b9a1-6776d78ba18c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:12.454401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ef2e3-5983-47e0-8216-28ad1eed29d1 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:12.472310 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967987d8-71ed-4824-afb2-f28e470ddc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:12.473297 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967987d8-71ed-4824-afb2-f28e470ddc1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:12.473497 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967987d8-71ed-4824-afb2-f28e470ddc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:12.473773 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967987d8-71ed-4824-afb2-f28e470ddc1a admin admin] PATH_INFO: `/v3/users/ccc3c8fcd64b4b618d99a092228dd345` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:12.476504 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967987d8-71ed-4824-afb2-f28e470ddc1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccc3c8fcd64b4b618d99a092228dd345)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:12.478687 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967987d8-71ed-4824-afb2-f28e470ddc1a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:12.568017 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe03f9e-2b9f-4700-a9c2-a042c1ea8fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:12.569155 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe03f9e-2b9f-4700-a9c2-a042c1ea8fbc admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:12.569442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe03f9e-2b9f-4700-a9c2-a042c1ea8fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:12.569725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe03f9e-2b9f-4700-a9c2-a042c1ea8fbc admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:12.572882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe03f9e-2b9f-4700-a9c2-a042c1ea8fbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:12.576508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe03f9e-2b9f-4700-a9c2-a042c1ea8fbc admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:12.607403 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:12.634514 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-967987d8-71ed-4824-afb2-f28e470ddc1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccc3c8fcd64b4b618d99a092228dd345'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:12.635510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-967987d8-71ed-4824-afb2-f28e470ddc1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccc3c8fcd64b4b618d99a092228dd345'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:12.654704 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 852/1686] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:12 2022] POST /identity/v3/users => generated 285 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:12.657054 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849513d-2a8d-4c51-b672-d619f3e1eb30 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:12.657817 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849513d-2a8d-4c51-b672-d619f3e1eb30 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:12.658921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849513d-2a8d-4c51-b672-d619f3e1eb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:12.665901 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-967987d8-71ed-4824-afb2-f28e470ddc1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccc3c8fcd64b4b618d99a092228dd345'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:12.683195 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-967987d8-71ed-4824-afb2-f28e470ddc1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccc3c8fcd64b4b618d99a092228dd345'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:12.685737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-967987d8-71ed-4824-afb2-f28e470ddc1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccc3c8fcd64b4b618d99a092228dd345'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:12.716527 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b849513d-2a8d-4c51-b672-d619f3e1eb30 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:12.718605 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b849513d-2a8d-4c51-b672-d619f3e1eb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:12.733276 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 835/1687] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:12 2022] DELETE /identity/v3/users/ccc3c8fcd64b4b618d99a092228dd345 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:12.733565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967987d8-71ed-4824-afb2-f28e470ddc1a admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:12.737303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-967987d8-71ed-4824-afb2-f28e470ddc1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:12.745513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 853/1688] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:12.745513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849513d-2a8d-4c51-b672-d619f3e1eb30 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:12.751678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b849513d-2a8d-4c51-b672-d619f3e1eb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:12.791755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:12.794344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849513d-2a8d-4c51-b672-d619f3e1eb30 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:12.944727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967987d8-71ed-4824-afb2-f28e470ddc1a admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:12.955492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f16c3c-23f5-4f37-b4ce-e4716e333d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:12.957505 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f16c3c-23f5-4f37-b4ce-e4716e333d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:12.957845 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f16c3c-23f5-4f37-b4ce-e4716e333d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:12.958131 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f16c3c-23f5-4f37-b4ce-e4716e333d3d admin admin] PATH_INFO: `/v3/projects/6c08311d50d54d4ead8b533530ae3dc9` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:12.959662 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f16c3c-23f5-4f37-b4ce-e4716e333d3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c08311d50d54d4ead8b533530ae3dc9)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:12.962480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f16c3c-23f5-4f37-b4ce-e4716e333d3d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:13.006016 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735ed34-0f1f-48f1-9709-249d8b24631f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:13.006809 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735ed34-0f1f-48f1-9709-249d8b24631f admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:13.007096 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735ed34-0f1f-48f1-9709-249d8b24631f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:13.007373 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735ed34-0f1f-48f1-9709-249d8b24631f admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:13.017257 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735ed34-0f1f-48f1-9709-249d8b24631f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:13.028961 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735ed34-0f1f-48f1-9709-249d8b24631f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:13.055241 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a0f16c3c-23f5-4f37-b4ce-e4716e333d3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c08311d50d54d4ead8b533530ae3dc9', 'request_id': 'req-a0f16c3c-23f5-4f37-b4ce-e4716e333d3d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:13.057453 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 854/1689] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:13.057831 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735ed34-0f1f-48f1-9709-249d8b24631f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:13.065477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a0f16c3c-23f5-4f37-b4ce-e4716e333d3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c08311d50d54d4ead8b533530ae3dc9', 'request_id': 'req-a0f16c3c-23f5-4f37-b4ce-e4716e333d3d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:13.087128 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735ed34-0f1f-48f1-9709-249d8b24631f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:13.090112 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a0f16c3c-23f5-4f37-b4ce-e4716e333d3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c08311d50d54d4ead8b533530ae3dc9', 'request_id': 'req-a0f16c3c-23f5-4f37-b4ce-e4716e333d3d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:13.107942 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a0f16c3c-23f5-4f37-b4ce-e4716e333d3d admin admin] The token cache is being invalidate because project 6c08311d50d54d4ead8b533530ae3dc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:13.108252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a0f16c3c-23f5-4f37-b4ce-e4716e333d3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:13.109242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a0f16c3c-23f5-4f37-b4ce-e4716e333d3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:13.118213 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 836/1690] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:12 2022] DELETE /identity/v3/projects/6c08311d50d54d4ead8b533530ae3dc9 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:13.122017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720b947-bbf8-41e5-8b70-a287953dbb26 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:13.122295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720b947-bbf8-41e5-8b70-a287953dbb26 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:13.122578 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720b947-bbf8-41e5-8b70-a287953dbb26 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:13.130329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 837/1691] 10.222.0.13 () {54 vars in 912 bytes} [Sat Jun 25 16:06:13 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:06:13.132008 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720b947-bbf8-41e5-8b70-a287953dbb26 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:13.134049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7720b947-bbf8-41e5-8b70-a287953dbb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:13.278741 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b5b38-8d6f-4289-ac93-dda8e9c873b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:13.279912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b5b38-8d6f-4289-ac93-dda8e9c873b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:13.280197 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b5b38-8d6f-4289-ac93-dda8e9c873b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:13.280494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b5b38-8d6f-4289-ac93-dda8e9c873b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:13.281237 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b5b38-8d6f-4289-ac93-dda8e9c873b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:06:13.281545 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b5b38-8d6f-4289-ac93-dda8e9c873b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:13.283155 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b5b38-8d6f-4289-ac93-dda8e9c873b2 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:13.295877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720b947-bbf8-41e5-8b70-a287953dbb26 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:13.303904 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131acc9-5fb5-4bb7-b17f-f486951f34ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:13.305328 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131acc9-5fb5-4bb7-b17f-f486951f34ae admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:13.306060 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131acc9-5fb5-4bb7-b17f-f486951f34ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:13.306477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131acc9-5fb5-4bb7-b17f-f486951f34ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:13.308136 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131acc9-5fb5-4bb7-b17f-f486951f34ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:13.309317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 855/1692] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:06:13.310322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b5b38-8d6f-4289-ac93-dda8e9c873b2 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:13.310454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131acc9-5fb5-4bb7-b17f-f486951f34ae admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:13.315312 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e17b5b38-8d6f-4289-ac93-dda8e9c873b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:13.322884 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 838/1693] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:13.324973 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d9aad-bd2b-42a7-a06a-84acc3cf84c9 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:13.325841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d9aad-bd2b-42a7-a06a-84acc3cf84c9 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:13.326835 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d9aad-bd2b-42a7-a06a-84acc3cf84c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:13.330424 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b5b38-8d6f-4289-ac93-dda8e9c873b2 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:13.337638 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a5131-d32f-4a78-83e5-abdf4f0b8836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:13.338295 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a5131-d32f-4a78-83e5-abdf4f0b8836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:13.338725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a5131-d32f-4a78-83e5-abdf4f0b8836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:13.339134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a5131-d32f-4a78-83e5-abdf4f0b8836 admin admin] PATH_INFO: `/v3/users/7fc1d0c6d5da470893e6e190b890369d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:13.340735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a5131-d32f-4a78-83e5-abdf4f0b8836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fc1d0c6d5da470893e6e190b890369d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:13.342922 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a5131-d32f-4a78-83e5-abdf4f0b8836 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:13.398335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f40d9aad-bd2b-42a7-a06a-84acc3cf84c9 None None] MFA Rules not processed for user `aaffb9f0bf714822829455c0487376ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:13.400101 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f40d9aad-bd2b-42a7-a06a-84acc3cf84c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:13.426987 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4b2a5131-d32f-4a78-83e5-abdf4f0b8836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fc1d0c6d5da470893e6e190b890369d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:13.433246 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:13.439794 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4b2a5131-d32f-4a78-83e5-abdf4f0b8836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fc1d0c6d5da470893e6e190b890369d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:13.440866 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4b2a5131-d32f-4a78-83e5-abdf4f0b8836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fc1d0c6d5da470893e6e190b890369d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:13.451545 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4b2a5131-d32f-4a78-83e5-abdf4f0b8836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fc1d0c6d5da470893e6e190b890369d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:13.452432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4b2a5131-d32f-4a78-83e5-abdf4f0b8836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fc1d0c6d5da470893e6e190b890369d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:13.459943 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:13.477620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 839/1694] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Jun 25 16:06:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:13.478632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d9aad-bd2b-42a7-a06a-84acc3cf84c9 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:13.481309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f40d9aad-bd2b-42a7-a06a-84acc3cf84c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:13.493990 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 856/1695] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:13 2022] DELETE /identity/v3/users/7fc1d0c6d5da470893e6e190b890369d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:13.495218 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a5131-d32f-4a78-83e5-abdf4f0b8836 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:13.497371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4b2a5131-d32f-4a78-83e5-abdf4f0b8836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:13.501933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d9aad-bd2b-42a7-a06a-84acc3cf84c9 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:13.591128 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c776dd-8643-4d73-9f38-31ba6958bab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:13.592045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c776dd-8643-4d73-9f38-31ba6958bab9 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:13.592525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c776dd-8643-4d73-9f38-31ba6958bab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:13.592935 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c776dd-8643-4d73-9f38-31ba6958bab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:13.595161 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c776dd-8643-4d73-9f38-31ba6958bab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:13.598535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c776dd-8643-4d73-9f38-31ba6958bab9 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:13.643939 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a5131-d32f-4a78-83e5-abdf4f0b8836 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:13.650560 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 840/1696] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:13 2022] POST /identity/v3/projects => generated 369 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:13.651935 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c776dd-8643-4d73-9f38-31ba6958bab9 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:13.654801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-21c776dd-8643-4d73-9f38-31ba6958bab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:13.667878 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced00952-1085-45eb-9092-76a213088bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:13.673772 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced00952-1085-45eb-9092-76a213088bf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:13.674676 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced00952-1085-45eb-9092-76a213088bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:13.675758 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced00952-1085-45eb-9092-76a213088bf7 admin admin] PATH_INFO: `/v3/projects/08002b21aa9d48dba64d381ac0ab01ea/users/1fbeb73840ef464390de23eb3ce36497/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:13.708267 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced00952-1085-45eb-9092-76a213088bf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08002b21aa9d48dba64d381ac0ab01ea, user_id=1fbeb73840ef464390de23eb3ce36497, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:13.711942 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced00952-1085-45eb-9092-76a213088bf7 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:13.743763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c776dd-8643-4d73-9f38-31ba6958bab9 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:13.755410 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 857/1697] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:13 2022] PUT /identity/v3/projects/08002b21aa9d48dba64d381ac0ab01ea/users/1fbeb73840ef464390de23eb3ce36497/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:13.756752 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced00952-1085-45eb-9092-76a213088bf7 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:13.818742 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62965ae9-b0b6-441f-a72a-90a0f1e3bdc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:13.819568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62965ae9-b0b6-441f-a72a-90a0f1e3bdc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:13.819861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62965ae9-b0b6-441f-a72a-90a0f1e3bdc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:13.820125 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62965ae9-b0b6-441f-a72a-90a0f1e3bdc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:13.834646 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62965ae9-b0b6-441f-a72a-90a0f1e3bdc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:13.836899 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62965ae9-b0b6-441f-a72a-90a0f1e3bdc9 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:13.875672 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 841/1698] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:13.878938 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44a30b-de7a-4947-89ac-8ead414ef751 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:13.879413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44a30b-de7a-4947-89ac-8ead414ef751 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:13.880081 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44a30b-de7a-4947-89ac-8ead414ef751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:13.927102 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-bc44a30b-de7a-4947-89ac-8ead414ef751 None None] MFA Rules not processed for user `1fbeb73840ef464390de23eb3ce36497`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:13.929135 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bc44a30b-de7a-4947-89ac-8ead414ef751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:13.954819 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:13.958240 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced00952-1085-45eb-9092-76a213088bf7 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:13.979252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9266438-ad9b-4c9c-a412-17a3bd9927b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:13.980055 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9266438-ad9b-4c9c-a412-17a3bd9927b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:13.981973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9266438-ad9b-4c9c-a412-17a3bd9927b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:13.982331 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9266438-ad9b-4c9c-a412-17a3bd9927b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:13.983666 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9266438-ad9b-4c9c-a412-17a3bd9927b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:13.986135 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9266438-ad9b-4c9c-a412-17a3bd9927b2 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:14.026008 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 858/1699] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:13 2022] POST /identity/v3/users => generated 285 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:14.030703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9266438-ad9b-4c9c-a412-17a3bd9927b2 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:14.066562 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9266438-ad9b-4c9c-a412-17a3bd9927b2 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:14.081992 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb226e-6ad0-4e44-abbc-00cfa4d316ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:14.082876 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb226e-6ad0-4e44-abbc-00cfa4d316ec admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:14.083220 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb226e-6ad0-4e44-abbc-00cfa4d316ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:14.083433 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb226e-6ad0-4e44-abbc-00cfa4d316ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:14.085204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb226e-6ad0-4e44-abbc-00cfa4d316ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:14.088766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb226e-6ad0-4e44-abbc-00cfa4d316ec admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:14.094193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 842/1700] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:13 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:14.097108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44a30b-de7a-4947-89ac-8ead414ef751 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:14.128658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 859/1701] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:14.131859 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb226e-6ad0-4e44-abbc-00cfa4d316ec admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:14.141955 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44a30b-de7a-4947-89ac-8ead414ef751 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:14.166809 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb226e-6ad0-4e44-abbc-00cfa4d316ec admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:14.167526 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e8d85-8402-42c8-9b9c-f6180ad9d1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:14.169016 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e8d85-8402-42c8-9b9c-f6180ad9d1bd admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:14.182923 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b09b9-3182-47e6-bfaa-9d95407b3ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:14.183771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b09b9-3182-47e6-bfaa-9d95407b3ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:14.184050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b09b9-3182-47e6-bfaa-9d95407b3ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:14.184329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b09b9-3182-47e6-bfaa-9d95407b3ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:14.185892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b09b9-3182-47e6-bfaa-9d95407b3ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:14.190187 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b09b9-3182-47e6-bfaa-9d95407b3ae1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:14.192831 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e8d85-8402-42c8-9b9c-f6180ad9d1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:14.192831 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e8d85-8402-42c8-9b9c-f6180ad9d1bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:14.193426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e8d85-8402-42c8-9b9c-f6180ad9d1bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:14.207503 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e8d85-8402-42c8-9b9c-f6180ad9d1bd admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:14.236985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 860/1702] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:14.238257 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b09b9-3182-47e6-bfaa-9d95407b3ae1 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:14.260260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b09b9-3182-47e6-bfaa-9d95407b3ae1 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:14.276559 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41861ad-c056-47f9-8050-85854d945e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:14.278743 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41861ad-c056-47f9-8050-85854d945e33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:14.279027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41861ad-c056-47f9-8050-85854d945e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:14.280673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41861ad-c056-47f9-8050-85854d945e33 admin admin] PATH_INFO: `/v3/projects/6fc72eafb8684a9dab826b0c2e7f3847/users/bd56b922b0e14da1a299b5052cd8bc95/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:14.302800 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 843/1703] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:14 2022] POST /identity/v3/projects => generated 371 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:14.304467 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e8d85-8402-42c8-9b9c-f6180ad9d1bd admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:14.329678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41861ad-c056-47f9-8050-85854d945e33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fc72eafb8684a9dab826b0c2e7f3847, user_id=bd56b922b0e14da1a299b5052cd8bc95, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:14.338120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41861ad-c056-47f9-8050-85854d945e33 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:14.385072 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 861/1704] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:14 2022] PUT /identity/v3/projects/6fc72eafb8684a9dab826b0c2e7f3847/users/bd56b922b0e14da1a299b5052cd8bc95/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:14.386132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41861ad-c056-47f9-8050-85854d945e33 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:14.393828 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c41861ad-c056-47f9-8050-85854d945e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:14.402371 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e8d85-8402-42c8-9b9c-f6180ad9d1bd admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:14.422454 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:14.531565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41861ad-c056-47f9-8050-85854d945e33 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:14.543079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee633ca-75c3-4fa5-9f41-dcb79681ed5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:14.544000 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee633ca-75c3-4fa5-9f41-dcb79681ed5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:14.544283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee633ca-75c3-4fa5-9f41-dcb79681ed5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:14.544589 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee633ca-75c3-4fa5-9f41-dcb79681ed5b admin admin] PATH_INFO: `/v3/users/33533f18b16f48e08babc4c07956a0d6` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:14.548675 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee633ca-75c3-4fa5-9f41-dcb79681ed5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=33533f18b16f48e08babc4c07956a0d6)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:14.550636 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee633ca-75c3-4fa5-9f41-dcb79681ed5b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:14.551510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c611261-ad10-4661-8e06-29fa409f9af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:14.552807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c611261-ad10-4661-8e06-29fa409f9af3 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:14.553282 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c611261-ad10-4661-8e06-29fa409f9af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:14.553739 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c611261-ad10-4661-8e06-29fa409f9af3 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:14.563463 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c611261-ad10-4661-8e06-29fa409f9af3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:14.565639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c611261-ad10-4661-8e06-29fa409f9af3 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:14.611387 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 844/1705] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:14 2022] POST /identity/v3/users => generated 292 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:14.612883 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f6e66-88a2-4603-bd1a-b7cb76b9fb0c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:14.613232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f6e66-88a2-4603-bd1a-b7cb76b9fb0c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:14.613572 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f6e66-88a2-4603-bd1a-b7cb76b9fb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:14.674096 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-0ee633ca-75c3-4fa5-9f41-dcb79681ed5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33533f18b16f48e08babc4c07956a0d6'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:14.691273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-0ee633ca-75c3-4fa5-9f41-dcb79681ed5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33533f18b16f48e08babc4c07956a0d6'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:14.692600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-0ee633ca-75c3-4fa5-9f41-dcb79681ed5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33533f18b16f48e08babc4c07956a0d6'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:14.704376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-0b1f6e66-88a2-4603-bd1a-b7cb76b9fb0c None None] MFA Rules not processed for user `bd56b922b0e14da1a299b5052cd8bc95`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:14.705779 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-0b1f6e66-88a2-4603-bd1a-b7cb76b9fb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:14.709103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-0ee633ca-75c3-4fa5-9f41-dcb79681ed5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33533f18b16f48e08babc4c07956a0d6'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:14.714828 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-0ee633ca-75c3-4fa5-9f41-dcb79681ed5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33533f18b16f48e08babc4c07956a0d6'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:14.739556 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 862/1706] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:14 2022] DELETE /identity/v3/users/33533f18b16f48e08babc4c07956a0d6 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:14.740679 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee633ca-75c3-4fa5-9f41-dcb79681ed5b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:14.743615 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0ee633ca-75c3-4fa5-9f41-dcb79681ed5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:14.753012 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:14.881931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee633ca-75c3-4fa5-9f41-dcb79681ed5b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:14.896116 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 845/1707] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:14 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:14.897435 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f6e66-88a2-4603-bd1a-b7cb76b9fb0c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:14.904018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d14f9e-7d94-4800-90bc-2d6d33abc4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:14.904463 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-0b1f6e66-88a2-4603-bd1a-b7cb76b9fb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:14.906506 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d14f9e-7d94-4800-90bc-2d6d33abc4a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:14.909043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d14f9e-7d94-4800-90bc-2d6d33abc4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:14.909391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d14f9e-7d94-4800-90bc-2d6d33abc4a5 admin admin] PATH_INFO: `/v3/projects/d14e0266d46e4e8ebc64a36f20766c6a` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:14.913322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d14f9e-7d94-4800-90bc-2d6d33abc4a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d14e0266d46e4e8ebc64a36f20766c6a)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:14.918881 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d14f9e-7d94-4800-90bc-2d6d33abc4a5 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:14.954150 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f6e66-88a2-4603-bd1a-b7cb76b9fb0c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:14.977275 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35deb216-41e7-433c-8d97-471ac873f63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:14.978065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35deb216-41e7-433c-8d97-471ac873f63c admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:14.978417 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35deb216-41e7-433c-8d97-471ac873f63c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:14.979352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35deb216-41e7-433c-8d97-471ac873f63c admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:14.980182 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35deb216-41e7-433c-8d97-471ac873f63c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:14.981915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35deb216-41e7-433c-8d97-471ac873f63c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:15.016443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 846/1708] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:15.024168 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cfcf1-9651-46cd-9383-e722989f5cf8 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:15.024525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cfcf1-9651-46cd-9383-e722989f5cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:15.024898 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cfcf1-9651-46cd-9383-e722989f5cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:15.042820 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:15.062238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a4d14f9e-7d94-4800-90bc-2d6d33abc4a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd14e0266d46e4e8ebc64a36f20766c6a', 'request_id': 'req-a4d14f9e-7d94-4800-90bc-2d6d33abc4a5'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:15.070260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a4d14f9e-7d94-4800-90bc-2d6d33abc4a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd14e0266d46e4e8ebc64a36f20766c6a', 'request_id': 'req-a4d14f9e-7d94-4800-90bc-2d6d33abc4a5'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:15.078837 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a4d14f9e-7d94-4800-90bc-2d6d33abc4a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd14e0266d46e4e8ebc64a36f20766c6a', 'request_id': 'req-a4d14f9e-7d94-4800-90bc-2d6d33abc4a5'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:15.093600 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-cd1cfcf1-9651-46cd-9383-e722989f5cf8 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:15.095032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-cd1cfcf1-9651-46cd-9383-e722989f5cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:15.139154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a4d14f9e-7d94-4800-90bc-2d6d33abc4a5 admin admin] The token cache is being invalidate because project d14e0266d46e4e8ebc64a36f20766c6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:15.139462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a4d14f9e-7d94-4800-90bc-2d6d33abc4a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:15.141156 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a4d14f9e-7d94-4800-90bc-2d6d33abc4a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:15.146445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 863/1709] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:14 2022] DELETE /identity/v3/projects/d14e0266d46e4e8ebc64a36f20766c6a => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:15.148699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d14f9e-7d94-4800-90bc-2d6d33abc4a5 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:15.155177 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a4d14f9e-7d94-4800-90bc-2d6d33abc4a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:15.267215 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 847/1710] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:15.268168 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cfcf1-9651-46cd-9383-e722989f5cf8 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:15.269720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-cd1cfcf1-9651-46cd-9383-e722989f5cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:15.291193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cfcf1-9651-46cd-9383-e722989f5cf8 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:15.306243 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6404e8a-ef81-4c32-bd94-9695b0edd9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:15.308287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6404e8a-ef81-4c32-bd94-9695b0edd9fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:15.308287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6404e8a-ef81-4c32-bd94-9695b0edd9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:15.308287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6404e8a-ef81-4c32-bd94-9695b0edd9fb admin admin] PATH_INFO: `/v3/projects/7d22394f24e84c45b5c1e696eadcf9f2/users/d8323c7522cd4868a9d2712db4344cd1/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:15.311323 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d14f9e-7d94-4800-90bc-2d6d33abc4a5 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:15.331920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c3f54-1f60-4bce-a2cf-33b3c3e13186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:15.332849 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c3f54-1f60-4bce-a2cf-33b3c3e13186 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:15.333135 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c3f54-1f60-4bce-a2cf-33b3c3e13186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:15.333410 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c3f54-1f60-4bce-a2cf-33b3c3e13186 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:15.334287 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c3f54-1f60-4bce-a2cf-33b3c3e13186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:15.336505 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c3f54-1f60-4bce-a2cf-33b3c3e13186 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:15.360479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6404e8a-ef81-4c32-bd94-9695b0edd9fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d22394f24e84c45b5c1e696eadcf9f2, user_id=d8323c7522cd4868a9d2712db4344cd1, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:15.366308 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6404e8a-ef81-4c32-bd94-9695b0edd9fb admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:15.391123 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 848/1711] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:15 2022] PUT /identity/v3/projects/7d22394f24e84c45b5c1e696eadcf9f2/users/d8323c7522cd4868a9d2712db4344cd1/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:15.392314 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6404e8a-ef81-4c32-bd94-9695b0edd9fb admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:15.395103 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a6404e8a-ef81-4c32-bd94-9695b0edd9fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:15.405937 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 864/1712] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:15 2022] POST /identity/v3/projects => generated 367 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:15.406709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c3f54-1f60-4bce-a2cf-33b3c3e13186 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:15.411940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1d6c3f54-1f60-4bce-a2cf-33b3c3e13186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:15.489176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6404e8a-ef81-4c32-bd94-9695b0edd9fb admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:15.529887 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:15.568985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c3f54-1f60-4bce-a2cf-33b3c3e13186 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:15.578781 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5eb5b-920b-49a5-8821-18dc0fc4b40a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:15.581552 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5eb5b-920b-49a5-8821-18dc0fc4b40a service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:15.583602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5eb5b-920b-49a5-8821-18dc0fc4b40a service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:15.584534 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5eb5b-920b-49a5-8821-18dc0fc4b40a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:15.605688 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42f1b2-c463-45d2-a40c-7329a6deb946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:15.607957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42f1b2-c463-45d2-a40c-7329a6deb946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:15.609097 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42f1b2-c463-45d2-a40c-7329a6deb946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:15.609475 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42f1b2-c463-45d2-a40c-7329a6deb946 admin admin] PATH_INFO: `/v3/users/2a3a7c47864947edab213a6e69059650` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:15.611712 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42f1b2-c463-45d2-a40c-7329a6deb946 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a3a7c47864947edab213a6e69059650)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:15.614926 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42f1b2-c463-45d2-a40c-7329a6deb946 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:15.708967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:15.713936 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7e42f1b2-c463-45d2-a40c-7329a6deb946 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a3a7c47864947edab213a6e69059650'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:15.715725 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7e42f1b2-c463-45d2-a40c-7329a6deb946 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a3a7c47864947edab213a6e69059650'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:15.715847 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5eb5b-920b-49a5-8821-18dc0fc4b40a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:15.717233 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5eb5b-920b-49a5-8821-18dc0fc4b40a service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:15.734644 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7e42f1b2-c463-45d2-a40c-7329a6deb946 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a3a7c47864947edab213a6e69059650'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:15.745222 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7e42f1b2-c463-45d2-a40c-7329a6deb946 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a3a7c47864947edab213a6e69059650'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:15.746029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7e42f1b2-c463-45d2-a40c-7329a6deb946 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a3a7c47864947edab213a6e69059650'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:15.755082 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 865/1713] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:15.756917 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5eb5b-920b-49a5-8821-18dc0fc4b40a service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:15.770081 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 849/1714] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:15 2022] DELETE /identity/v3/users/2a3a7c47864947edab213a6e69059650 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:15.771197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42f1b2-c463-45d2-a40c-7329a6deb946 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:15.772623 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7e42f1b2-c463-45d2-a40c-7329a6deb946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:15.784526 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5eb5b-920b-49a5-8821-18dc0fc4b40a service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:15.871806 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42f1b2-c463-45d2-a40c-7329a6deb946 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:15.876981 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01bbb75-8776-4577-835a-b5891a21b0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:15.878191 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bbb75-8776-4577-835a-b5891a21b0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:15.878191 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bbb75-8776-4577-835a-b5891a21b0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:15.878432 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bbb75-8776-4577-835a-b5891a21b0b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:15.879437 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01bbb75-8776-4577-835a-b5891a21b0b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:15.882222 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01bbb75-8776-4577-835a-b5891a21b0b4 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:15.920995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852c876-014f-4c40-9069-e76d1f6db844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:15.921762 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852c876-014f-4c40-9069-e76d1f6db844 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:15.922139 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852c876-014f-4c40-9069-e76d1f6db844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:15.922325 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852c876-014f-4c40-9069-e76d1f6db844 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:15.923004 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 850/1715] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:15 2022] POST /identity/v3/users => generated 290 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:15.923111 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852c876-014f-4c40-9069-e76d1f6db844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:15.924781 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852c876-014f-4c40-9069-e76d1f6db844 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:15.927017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c87dae-1c41-4c41-944c-b5ac912f9741 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:15.927279 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c87dae-1c41-4c41-944c-b5ac912f9741 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:15.927562 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c87dae-1c41-4c41-944c-b5ac912f9741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:15.957559 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 866/1716] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:15.958627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852c876-014f-4c40-9069-e76d1f6db844 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:15.975865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852c876-014f-4c40-9069-e76d1f6db844 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:15.984171 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631c2de-74ad-4eb8-89b2-2ab629f79ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:15.985111 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631c2de-74ad-4eb8-89b2-2ab629f79ece admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:15.985397 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631c2de-74ad-4eb8-89b2-2ab629f79ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:15.985750 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631c2de-74ad-4eb8-89b2-2ab629f79ece admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:15.986634 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631c2de-74ad-4eb8-89b2-2ab629f79ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:15.988597 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631c2de-74ad-4eb8-89b2-2ab629f79ece admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:16.009082 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 867/1717] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:16.009928 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631c2de-74ad-4eb8-89b2-2ab629f79ece admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:16.011960 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1631c2de-74ad-4eb8-89b2-2ab629f79ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:16.019731 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d9c87dae-1c41-4c41-944c-b5ac912f9741 None None] MFA Rules not processed for user `19140330ae664e45b85f47f321496191`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:16.021745 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d9c87dae-1c41-4c41-944c-b5ac912f9741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:16.026978 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631c2de-74ad-4eb8-89b2-2ab629f79ece admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:16.033181 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a9ae8-9331-485f-8d73-c38ebfa67f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:16.034441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a9ae8-9331-485f-8d73-c38ebfa67f35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:16.034886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a9ae8-9331-485f-8d73-c38ebfa67f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:16.035299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a9ae8-9331-485f-8d73-c38ebfa67f35 admin admin] PATH_INFO: `/v3/projects/7d22394f24e84c45b5c1e696eadcf9f2/users/d8323c7522cd4868a9d2712db4344cd1/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:16.038999 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a9ae8-9331-485f-8d73-c38ebfa67f35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d22394f24e84c45b5c1e696eadcf9f2, user_id=d8323c7522cd4868a9d2712db4344cd1, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:16.044235 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9a9ae8-9331-485f-8d73-c38ebfa67f35 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:16.068523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 868/1718] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:16 2022] PUT /identity/v3/projects/7d22394f24e84c45b5c1e696eadcf9f2/users/d8323c7522cd4868a9d2712db4344cd1/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:16.068523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a9ae8-9331-485f-8d73-c38ebfa67f35 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:16.108929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:16.245991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 851/1719] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:16.247277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a9ae8-9331-485f-8d73-c38ebfa67f35 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:16.250421 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a9c99-5797-4ef7-b56a-d18b4f065a74 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:16.251127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a9c99-5797-4ef7-b56a-d18b4f065a74 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:16.255021 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a9c99-5797-4ef7-b56a-d18b4f065a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:16.272007 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490d936-9b0d-4de7-8a0d-c30ce428e662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:16.273022 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490d936-9b0d-4de7-8a0d-c30ce428e662 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:16.273340 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490d936-9b0d-4de7-8a0d-c30ce428e662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:16.273617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490d936-9b0d-4de7-8a0d-c30ce428e662 admin admin] PATH_INFO: `/v3/projects/fc864821e72d4825943e0d28ee3e1b1c/users/a401d11c9ae446cc9418fa631e7dbe58/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:16.324762 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490d936-9b0d-4de7-8a0d-c30ce428e662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc864821e72d4825943e0d28ee3e1b1c, user_id=a401d11c9ae446cc9418fa631e7dbe58, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:16.330493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490d936-9b0d-4de7-8a0d-c30ce428e662 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:16.346742 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c48a9c99-5797-4ef7-b56a-d18b4f065a74 None None] MFA Rules not processed for user `d8323c7522cd4868a9d2712db4344cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:16.352637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c48a9c99-5797-4ef7-b56a-d18b4f065a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:16.384317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 869/1720] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:16 2022] PUT /identity/v3/projects/fc864821e72d4825943e0d28ee3e1b1c/users/a401d11c9ae446cc9418fa631e7dbe58/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:16.388161 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490d936-9b0d-4de7-8a0d-c30ce428e662 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:16.392350 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b490d936-9b0d-4de7-8a0d-c30ce428e662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:16.587136 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:16.649280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490d936-9b0d-4de7-8a0d-c30ce428e662 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:16.659921 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 852/1721] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:16 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:16.662529 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45c23b-30fd-408e-93e5-943509110ebe None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:16.663017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45c23b-30fd-408e-93e5-943509110ebe None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:16.663498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45c23b-30fd-408e-93e5-943509110ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:16.665672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d5604-6894-4bd2-aa8e-31db28375580 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:16.667631 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d5604-6894-4bd2-aa8e-31db28375580 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:16.667934 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d5604-6894-4bd2-aa8e-31db28375580 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:16.668216 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d5604-6894-4bd2-aa8e-31db28375580 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:16.717504 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-8e45c23b-30fd-408e-93e5-943509110ebe None None] MFA Rules not processed for user `be06179595ef450a917d71fe4e6b2c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:16.718877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8e45c23b-30fd-408e-93e5-943509110ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:16.787355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d5604-6894-4bd2-aa8e-31db28375580 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:16.789123 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d5604-6894-4bd2-aa8e-31db28375580 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:16.809500 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 870/1722] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:16.812474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d5604-6894-4bd2-aa8e-31db28375580 service nova] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:16.815936 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-894d5604-6894-4bd2-aa8e-31db28375580 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:16.859876 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 853/1723] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:16.859876 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45c23b-30fd-408e-93e5-943509110ebe None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:17.042277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:17.044678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d5604-6894-4bd2-aa8e-31db28375580 service nova] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:17.071280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37835f1-012e-469e-8da3-3e9276adcf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:17.073342 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37835f1-012e-469e-8da3-3e9276adcf96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:17.073630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37835f1-012e-469e-8da3-3e9276adcf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:17.077343 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37835f1-012e-469e-8da3-3e9276adcf96 admin admin] PATH_INFO: `/v3/projects/e392401b028c4ef29f9959a543445c69` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:17.083584 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37835f1-012e-469e-8da3-3e9276adcf96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e392401b028c4ef29f9959a543445c69)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:17.088196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37835f1-012e-469e-8da3-3e9276adcf96 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:17.112967 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45c23b-30fd-408e-93e5-943509110ebe None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:17.129110 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940ab87-fa14-4461-89a8-4cbb1be879ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:17.129819 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940ab87-fa14-4461-89a8-4cbb1be879ef admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:17.130100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940ab87-fa14-4461-89a8-4cbb1be879ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:17.130558 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940ab87-fa14-4461-89a8-4cbb1be879ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:17.131319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940ab87-fa14-4461-89a8-4cbb1be879ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:17.133143 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940ab87-fa14-4461-89a8-4cbb1be879ef admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:17.176699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 854/1724] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:17.182225 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6beb8f4-1cb2-4488-9348-3118e31d9b20 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:17.183346 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6beb8f4-1cb2-4488-9348-3118e31d9b20 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:17.184202 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6beb8f4-1cb2-4488-9348-3118e31d9b20 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:17.188008 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 855/1725] 10.222.0.13 () {54 vars in 960 bytes} [Sat Jun 25 16:06:17 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:06:17.189766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6beb8f4-1cb2-4488-9348-3118e31d9b20 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:17.197147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d6beb8f4-1cb2-4488-9348-3118e31d9b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:17.229628 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c37835f1-012e-469e-8da3-3e9276adcf96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e392401b028c4ef29f9959a543445c69', 'request_id': 'req-c37835f1-012e-469e-8da3-3e9276adcf96'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:17.252653 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c37835f1-012e-469e-8da3-3e9276adcf96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e392401b028c4ef29f9959a543445c69', 'request_id': 'req-c37835f1-012e-469e-8da3-3e9276adcf96'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:17.280245 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c37835f1-012e-469e-8da3-3e9276adcf96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e392401b028c4ef29f9959a543445c69', 'request_id': 'req-c37835f1-012e-469e-8da3-3e9276adcf96'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:17.313628 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c37835f1-012e-469e-8da3-3e9276adcf96 admin admin] The token cache is being invalidate because project e392401b028c4ef29f9959a543445c69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:17.314364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c37835f1-012e-469e-8da3-3e9276adcf96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:17.316726 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c37835f1-012e-469e-8da3-3e9276adcf96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:17.322502 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 871/1726] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:16 2022] DELETE /identity/v3/projects/e392401b028c4ef29f9959a543445c69 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:17.325009 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db08e8-1838-45eb-a197-4a867e049963 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:17.326241 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db08e8-1838-45eb-a197-4a867e049963 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:17.328488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db08e8-1838-45eb-a197-4a867e049963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:17.392787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-07db08e8-1838-45eb-a197-4a867e049963 None None] MFA Rules not processed for user `8a3137566b004edb909d4e3c702a7f59`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:17.394102 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-07db08e8-1838-45eb-a197-4a867e049963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:17.403487 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6beb8f4-1cb2-4488-9348-3118e31d9b20 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:17.424709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:17.426852 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819142a6-cbad-468a-aa89-926f7441d797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:17.428141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819142a6-cbad-468a-aa89-926f7441d797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:17.428732 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819142a6-cbad-468a-aa89-926f7441d797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:17.429462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819142a6-cbad-468a-aa89-926f7441d797 admin admin] PATH_INFO: `/v3/projects/8ef564f495b4489e98a852ad9fb7c5bd` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:17.432354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819142a6-cbad-468a-aa89-926f7441d797 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ef564f495b4489e98a852ad9fb7c5bd)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:17.437688 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819142a6-cbad-468a-aa89-926f7441d797 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:17.516259 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 872/1727] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:17.517552 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db08e8-1838-45eb-a197-4a867e049963 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:17.521580 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-07db08e8-1838-45eb-a197-4a867e049963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:17.568468 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-819142a6-cbad-468a-aa89-926f7441d797 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ef564f495b4489e98a852ad9fb7c5bd', 'request_id': 'req-819142a6-cbad-468a-aa89-926f7441d797'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:17.590252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-819142a6-cbad-468a-aa89-926f7441d797 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ef564f495b4489e98a852ad9fb7c5bd', 'request_id': 'req-819142a6-cbad-468a-aa89-926f7441d797'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:17.598299 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-819142a6-cbad-468a-aa89-926f7441d797 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ef564f495b4489e98a852ad9fb7c5bd', 'request_id': 'req-819142a6-cbad-468a-aa89-926f7441d797'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:17.629565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-819142a6-cbad-468a-aa89-926f7441d797 admin admin] The token cache is being invalidate because project 8ef564f495b4489e98a852ad9fb7c5bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:17.629565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-819142a6-cbad-468a-aa89-926f7441d797 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:17.630189 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-819142a6-cbad-468a-aa89-926f7441d797 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:17.634130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 856/1728] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:17 2022] DELETE /identity/v3/projects/8ef564f495b4489e98a852ad9fb7c5bd => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:17.637469 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819142a6-cbad-468a-aa89-926f7441d797 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:17.637469 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-819142a6-cbad-468a-aa89-926f7441d797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:17.667871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db08e8-1838-45eb-a197-4a867e049963 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:17.802629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfe117-454e-42b0-a929-b84c317555da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:17.803564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfe117-454e-42b0-a929-b84c317555da service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:17.804012 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfe117-454e-42b0-a929-b84c317555da service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:17.804292 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfe117-454e-42b0-a929-b84c317555da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:17.806223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819142a6-cbad-468a-aa89-926f7441d797 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:17.809787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-04bfe117-454e-42b0-a929-b84c317555da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:17.822627 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ec228-2dda-4ec1-ae11-11ff4d8c72ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:17.823634 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ec228-2dda-4ec1-ae11-11ff4d8c72ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:17.824007 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ec228-2dda-4ec1-ae11-11ff4d8c72ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:17.824375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ec228-2dda-4ec1-ae11-11ff4d8c72ec admin admin] PATH_INFO: `/v3/projects/a54ec262520c49838ffb02d54800597e` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:17.827588 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ec228-2dda-4ec1-ae11-11ff4d8c72ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=a54ec262520c49838ffb02d54800597e)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:17.829707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ec228-2dda-4ec1-ae11-11ff4d8c72ec admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:17.854647 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:17.939300 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-667ec228-2dda-4ec1-ae11-11ff4d8c72ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a54ec262520c49838ffb02d54800597e', 'request_id': 'req-667ec228-2dda-4ec1-ae11-11ff4d8c72ec'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:17.951672 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-667ec228-2dda-4ec1-ae11-11ff4d8c72ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a54ec262520c49838ffb02d54800597e', 'request_id': 'req-667ec228-2dda-4ec1-ae11-11ff4d8c72ec'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:17.965844 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-667ec228-2dda-4ec1-ae11-11ff4d8c72ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a54ec262520c49838ffb02d54800597e', 'request_id': 'req-667ec228-2dda-4ec1-ae11-11ff4d8c72ec'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:17.988159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-667ec228-2dda-4ec1-ae11-11ff4d8c72ec admin admin] The token cache is being invalidate because project a54ec262520c49838ffb02d54800597e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:17.989255 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-667ec228-2dda-4ec1-ae11-11ff4d8c72ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:17.993499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-667ec228-2dda-4ec1-ae11-11ff4d8c72ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:17.996445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 857/1729] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:17 2022] DELETE /identity/v3/projects/a54ec262520c49838ffb02d54800597e => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:18.001498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8432e-2802-4408-9d14-873c0245dfe6 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:18.002436 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8432e-2802-4408-9d14-873c0245dfe6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:18.002892 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8432e-2802-4408-9d14-873c0245dfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:18.034103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bfe117-454e-42b0-a929-b84c317555da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:18.035489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bfe117-454e-42b0-a929-b84c317555da service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:18.047527 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-02f8432e-2802-4408-9d14-873c0245dfe6 None None] MFA Rules not processed for user `be06179595ef450a917d71fe4e6b2c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:18.049361 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-02f8432e-2802-4408-9d14-873c0245dfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:18.070360 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 873/1730] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:17 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:18.072167 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfe117-454e-42b0-a929-b84c317555da service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:18.073707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-04bfe117-454e-42b0-a929-b84c317555da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:18.098819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfe117-454e-42b0-a929-b84c317555da service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:18.173392 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:18.227015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 858/1731] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:18.229239 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d9d05-8dee-425c-8ad8-3bdab71acd5a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:18.229537 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d9d05-8dee-425c-8ad8-3bdab71acd5a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:18.229818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d9d05-8dee-425c-8ad8-3bdab71acd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:18.266783 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a33d9d05-8dee-425c-8ad8-3bdab71acd5a None None] MFA Rules not processed for user `474c468a610140f8a5b18d5a1ed1d18e`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:18.268019 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a33d9d05-8dee-425c-8ad8-3bdab71acd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:18.282794 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee98428-df85-4e80-90ab-5655e9b6dfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:18.283956 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee98428-df85-4e80-90ab-5655e9b6dfea admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:18.284798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee98428-df85-4e80-90ab-5655e9b6dfea admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:18.285175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee98428-df85-4e80-90ab-5655e9b6dfea admin admin] PATH_INFO: `/v3/projects/fc864821e72d4825943e0d28ee3e1b1c/users/a401d11c9ae446cc9418fa631e7dbe58/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:18.290293 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee98428-df85-4e80-90ab-5655e9b6dfea admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc864821e72d4825943e0d28ee3e1b1c, user_id=a401d11c9ae446cc9418fa631e7dbe58, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:18.296074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee98428-df85-4e80-90ab-5655e9b6dfea admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:18.376131 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 874/1732] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:18 2022] PUT /identity/v3/projects/fc864821e72d4825943e0d28ee3e1b1c/users/a401d11c9ae446cc9418fa631e7dbe58/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:18.376131 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58123402-c8e8-41e3-8d50-7f91ea5b181f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:18.376131 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58123402-c8e8-41e3-8d50-7f91ea5b181f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:18.376131 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58123402-c8e8-41e3-8d50-7f91ea5b181f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:18.411645 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 859/1733] 10.222.0.13 () {58 vars in 1057 bytes} [Sat Jun 25 16:06:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:18.413031 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99dbe0-7610-496a-8154-52081235ebae None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:18.413422 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99dbe0-7610-496a-8154-52081235ebae None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:18.413707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99dbe0-7610-496a-8154-52081235ebae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:18.422793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-58123402-c8e8-41e3-8d50-7f91ea5b181f None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:18.424055 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-58123402-c8e8-41e3-8d50-7f91ea5b181f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:18.435767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-aa99dbe0-7610-496a-8154-52081235ebae None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:18.437239 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-aa99dbe0-7610-496a-8154-52081235ebae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:18.510827 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:18.553727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 860/1734] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:18.555554 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f14da-6fcb-4a93-827b-3db3f045c710 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:18.555843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f14da-6fcb-4a93-827b-3db3f045c710 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:18.556193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f14da-6fcb-4a93-827b-3db3f045c710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:18.578730 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d06f14da-6fcb-4a93-827b-3db3f045c710 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:18.581027 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d06f14da-6fcb-4a93-827b-3db3f045c710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:18.581513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 875/1735] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:18.585589 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef97e5-e002-4752-8112-d1eda72a9f90 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:18.587265 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef97e5-e002-4752-8112-d1eda72a9f90 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:18.588081 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef97e5-e002-4752-8112-d1eda72a9f90 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:18.592983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 876/1736] 10.222.0.13 () {54 vars in 912 bytes} [Sat Jun 25 16:06:18 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:06:18.594185 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef97e5-e002-4752-8112-d1eda72a9f90 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:18.599585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4cef97e5-e002-4752-8112-d1eda72a9f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:18.608874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 861/1737] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:18.610506 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ac3e5-e468-4934-b9b5-3a0ff136707e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:18.610847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ac3e5-e468-4934-b9b5-3a0ff136707e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:18.611134 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ac3e5-e468-4934-b9b5-3a0ff136707e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:18.659360 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-310ac3e5-e468-4934-b9b5-3a0ff136707e None None] MFA Rules not processed for user `a401d11c9ae446cc9418fa631e7dbe58`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:18.661031 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-310ac3e5-e468-4934-b9b5-3a0ff136707e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:18.871156 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 862/1738] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:18 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:18.872044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ac3e5-e468-4934-b9b5-3a0ff136707e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:18.958417 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef97e5-e002-4752-8112-d1eda72a9f90 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:18.967230 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263d0f3-9f96-4e80-b12b-c839a0d55539 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:18.968173 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263d0f3-9f96-4e80-b12b-c839a0d55539 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:18.968709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263d0f3-9f96-4e80-b12b-c839a0d55539 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:18.969242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263d0f3-9f96-4e80-b12b-c839a0d55539 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:18.971881 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d263d0f3-9f96-4e80-b12b-c839a0d55539 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:18.985859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ac3e5-e468-4934-b9b5-3a0ff136707e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:18.997103 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54146bd5-17af-48a6-b03c-c7fa5f63ab40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474c468a610140f8a5b18d5a1ed1d18e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:18.997829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54146bd5-17af-48a6-b03c-c7fa5f63ab40 service placement] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:18.998112 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54146bd5-17af-48a6-b03c-c7fa5f63ab40 service placement] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:18.998891 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54146bd5-17af-48a6-b03c-c7fa5f63ab40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:19.003972 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-54146bd5-17af-48a6-b03c-c7fa5f63ab40 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:19.041149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54146bd5-17af-48a6-b03c-c7fa5f63ab40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:19.043106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54146bd5-17af-48a6-b03c-c7fa5f63ab40 service placement] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:19.065608 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 863/1739] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:19.079703 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a818205-92f3-417e-8eec-b988fd244f7b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:19.084974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a818205-92f3-417e-8eec-b988fd244f7b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:19.085508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a818205-92f3-417e-8eec-b988fd244f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:19.105856 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d263d0f3-9f96-4e80-b12b-c839a0d55539 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:19.108076 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d263d0f3-9f96-4e80-b12b-c839a0d55539 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:19.120546 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2a818205-92f3-417e-8eec-b988fd244f7b None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:19.122875 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:19.125356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2a818205-92f3-417e-8eec-b988fd244f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:19.135932 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 877/1740] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:19.137380 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac64eb-dbbb-4d2f-a1b7-b3e4e38ac46d None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:19.137672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac64eb-dbbb-4d2f-a1b7-b3e4e38ac46d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:19.139071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac64eb-dbbb-4d2f-a1b7-b3e4e38ac46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:19.176916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-79ac64eb-dbbb-4d2f-a1b7-b3e4e38ac46d None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:19.179119 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-79ac64eb-dbbb-4d2f-a1b7-b3e4e38ac46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:19.181581 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 864/1741] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:19.194753 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf06e1b-c4be-4616-89d5-04be36077546 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:19.195015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf06e1b-c4be-4616-89d5-04be36077546 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:19.195565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf06e1b-c4be-4616-89d5-04be36077546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:19.196333 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 878/1742] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:19.202516 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8b744-e1f0-4b6d-bb9c-3d5df3ef6fcc None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:19.202804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8b744-e1f0-4b6d-bb9c-3d5df3ef6fcc None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:19.203087 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8b744-e1f0-4b6d-bb9c-3d5df3ef6fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:19.266408 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-46d8b744-e1f0-4b6d-bb9c-3d5df3ef6fcc None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:19.267997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-46d8b744-e1f0-4b6d-bb9c-3d5df3ef6fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:19.277124 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-adf06e1b-c4be-4616-89d5-04be36077546 None None] MFA Rules not processed for user `aaffb9f0bf714822829455c0487376ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:19.279016 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 879/1743] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:19.280016 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8b744-e1f0-4b6d-bb9c-3d5df3ef6fcc None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:19.281305 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-adf06e1b-c4be-4616-89d5-04be36077546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:19.281611 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-46d8b744-e1f0-4b6d-bb9c-3d5df3ef6fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:19.308799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8b744-e1f0-4b6d-bb9c-3d5df3ef6fcc None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:19.315547 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98baec-fa32-495e-b531-7f7e89feb10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:19.316486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98baec-fa32-495e-b531-7f7e89feb10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:19.317033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98baec-fa32-495e-b531-7f7e89feb10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:19.317411 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98baec-fa32-495e-b531-7f7e89feb10c admin admin] PATH_INFO: `/v3/users/1fbeb73840ef464390de23eb3ce36497` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:19.318951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98baec-fa32-495e-b531-7f7e89feb10c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fbeb73840ef464390de23eb3ce36497)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:19.321114 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98baec-fa32-495e-b531-7f7e89feb10c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:19.417807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9b98baec-fa32-495e-b531-7f7e89feb10c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fbeb73840ef464390de23eb3ce36497'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:19.440659 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:19.447498 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9b98baec-fa32-495e-b531-7f7e89feb10c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fbeb73840ef464390de23eb3ce36497'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:19.449618 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9b98baec-fa32-495e-b531-7f7e89feb10c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fbeb73840ef464390de23eb3ce36497'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:19.464271 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9b98baec-fa32-495e-b531-7f7e89feb10c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fbeb73840ef464390de23eb3ce36497'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:19.465986 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9b98baec-fa32-495e-b531-7f7e89feb10c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fbeb73840ef464390de23eb3ce36497'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:19.484548 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 865/1744] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Jun 25 16:06:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:19.487027 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf06e1b-c4be-4616-89d5-04be36077546 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:19.491641 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-adf06e1b-c4be-4616-89d5-04be36077546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:19.500898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 880/1745] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:19 2022] DELETE /identity/v3/users/1fbeb73840ef464390de23eb3ce36497 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:19.501931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98baec-fa32-495e-b531-7f7e89feb10c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:19.503355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9b98baec-fa32-495e-b531-7f7e89feb10c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:19.597616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98baec-fa32-495e-b531-7f7e89feb10c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:19.611480 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98375e-42cd-4b1b-a867-ecb3c8f1173a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:19.612256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98375e-42cd-4b1b-a867-ecb3c8f1173a admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:19.612707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98375e-42cd-4b1b-a867-ecb3c8f1173a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:19.613053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98375e-42cd-4b1b-a867-ecb3c8f1173a admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:19.613801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98375e-42cd-4b1b-a867-ecb3c8f1173a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:06:19.614154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98375e-42cd-4b1b-a867-ecb3c8f1173a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:19.615842 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98375e-42cd-4b1b-a867-ecb3c8f1173a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:19.638629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 881/1746] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:06:19.639509 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98375e-42cd-4b1b-a867-ecb3c8f1173a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:19.641750 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-be98375e-42cd-4b1b-a867-ecb3c8f1173a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:19.658423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98375e-42cd-4b1b-a867-ecb3c8f1173a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:19.664146 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcbc285-6dd4-4863-9d22-560c886ad12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:19.665047 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbc285-6dd4-4863-9d22-560c886ad12d admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:19.665766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbc285-6dd4-4863-9d22-560c886ad12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:19.665766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcbc285-6dd4-4863-9d22-560c886ad12d admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:19.666389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcbc285-6dd4-4863-9d22-560c886ad12d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:06:19.666755 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcbc285-6dd4-4863-9d22-560c886ad12d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:19.669492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcbc285-6dd4-4863-9d22-560c886ad12d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:19.684157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 882/1747] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:06:19.688771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcbc285-6dd4-4863-9d22-560c886ad12d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:19.690601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7bcbc285-6dd4-4863-9d22-560c886ad12d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:19.727435 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcbc285-6dd4-4863-9d22-560c886ad12d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:19.734941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25048a90-63ab-472d-a179-64ad71b77e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:19.735716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25048a90-63ab-472d-a179-64ad71b77e78 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:19.736001 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25048a90-63ab-472d-a179-64ad71b77e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:19.737729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25048a90-63ab-472d-a179-64ad71b77e78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:19.738609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25048a90-63ab-472d-a179-64ad71b77e78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:06:19.739344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25048a90-63ab-472d-a179-64ad71b77e78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:19.741102 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25048a90-63ab-472d-a179-64ad71b77e78 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:19.745072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf06e1b-c4be-4616-89d5-04be36077546 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:19.765467 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 883/1748] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:06:19.765467 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ffda9-ac78-48f3-b164-9591dfe8a231 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:19.765467 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ffda9-ac78-48f3-b164-9591dfe8a231 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:19.765467 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ffda9-ac78-48f3-b164-9591dfe8a231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:19.785626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-992ffda9-ac78-48f3-b164-9591dfe8a231 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:19.786452 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-992ffda9-ac78-48f3-b164-9591dfe8a231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:19.809685 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 884/1749] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:19.810774 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ffda9-ac78-48f3-b164-9591dfe8a231 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:19.812390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-992ffda9-ac78-48f3-b164-9591dfe8a231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:19.910396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:19.952942 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b707da-f9ee-4282-b996-ac53451f15b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:19.953888 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b707da-f9ee-4282-b996-ac53451f15b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:19.954384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b707da-f9ee-4282-b996-ac53451f15b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:19.955059 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b707da-f9ee-4282-b996-ac53451f15b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:19.958796 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b0b707da-f9ee-4282-b996-ac53451f15b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:20.014514 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ffda9-ac78-48f3-b164-9591dfe8a231 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:20.025274 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206d237-7aa8-409a-a365-be29817f9b4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:20.027398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206d237-7aa8-409a-a365-be29817f9b4a service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:20.027946 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206d237-7aa8-409a-a365-be29817f9b4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:20.029354 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206d237-7aa8-409a-a365-be29817f9b4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:20.069801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206d237-7aa8-409a-a365-be29817f9b4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:20.087281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206d237-7aa8-409a-a365-be29817f9b4a service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:20.114569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 885/1750] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:20.115612 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206d237-7aa8-409a-a365-be29817f9b4a service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:20.151330 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b707da-f9ee-4282-b996-ac53451f15b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:20.154743 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206d237-7aa8-409a-a365-be29817f9b4a service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:20.165330 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66d574-7558-480e-bc85-77e3121b0d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:20.166761 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b707da-f9ee-4282-b996-ac53451f15b4 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:20.170323 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66d574-7558-480e-bc85-77e3121b0d80 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:20.170630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66d574-7558-480e-bc85-77e3121b0d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:20.171014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66d574-7558-480e-bc85-77e3121b0d80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:20.174110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66d574-7558-480e-bc85-77e3121b0d80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:20.188020 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66d574-7558-480e-bc85-77e3121b0d80 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:20.232224 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 866/1751] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:19 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 727 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:20.233991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b707da-f9ee-4282-b996-ac53451f15b4 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:20.271574 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 886/1752] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:20 2022] POST /identity/v3/projects => generated 371 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:20.273201 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b707da-f9ee-4282-b996-ac53451f15b4 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:20.274397 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66d574-7558-480e-bc85-77e3121b0d80 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:20.449347 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:20.462661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9c2b6-f3d8-4409-bdeb-6274370111e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:20.463509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9c2b6-f3d8-4409-bdeb-6274370111e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:20.463807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9c2b6-f3d8-4409-bdeb-6274370111e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:20.464084 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9c2b6-f3d8-4409-bdeb-6274370111e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:20.464930 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9c2b6-f3d8-4409-bdeb-6274370111e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:20.466333 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66d574-7558-480e-bc85-77e3121b0d80 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:20.467253 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9c2b6-f3d8-4409-bdeb-6274370111e6 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:20.483378 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139f420-9234-4494-9373-ab5a8e6f4cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:20.484230 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139f420-9234-4494-9373-ab5a8e6f4cdf admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:20.484644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139f420-9234-4494-9373-ab5a8e6f4cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:20.485022 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139f420-9234-4494-9373-ab5a8e6f4cdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:20.486128 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139f420-9234-4494-9373-ab5a8e6f4cdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:20.488305 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139f420-9234-4494-9373-ab5a8e6f4cdf admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:20.517046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 867/1753] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:20 2022] POST /identity/v3/projects => generated 381 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:20.520894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9c2b6-f3d8-4409-bdeb-6274370111e6 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:20.524859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-16a9c2b6-f3d8-4409-bdeb-6274370111e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:20.560730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 887/1754] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:20 2022] POST /identity/v3/projects => generated 379 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:20.562562 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139f420-9234-4494-9373-ab5a8e6f4cdf admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:20.704991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9c2b6-f3d8-4409-bdeb-6274370111e6 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:20.761113 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139f420-9234-4494-9373-ab5a8e6f4cdf admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:20.780825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45472dc-69b9-4ec6-b26e-2e86d80eb36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:20.782755 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45472dc-69b9-4ec6-b26e-2e86d80eb36d admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:20.785286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45472dc-69b9-4ec6-b26e-2e86d80eb36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:20.785955 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45472dc-69b9-4ec6-b26e-2e86d80eb36d admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:20.787847 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45472dc-69b9-4ec6-b26e-2e86d80eb36d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:20.791143 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45472dc-69b9-4ec6-b26e-2e86d80eb36d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:20.857814 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495167bb-ef7f-476b-a115-e9c734106efa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:20.857814 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495167bb-ef7f-476b-a115-e9c734106efa service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:20.857814 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495167bb-ef7f-476b-a115-e9c734106efa service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:20.857814 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495167bb-ef7f-476b-a115-e9c734106efa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:20.858588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 888/1755] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:20 2022] POST /identity/v3/users => generated 286 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:20.858588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45472dc-69b9-4ec6-b26e-2e86d80eb36d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:20.858588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e45472dc-69b9-4ec6-b26e-2e86d80eb36d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:20.882552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495167bb-ef7f-476b-a115-e9c734106efa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:20.884910 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495167bb-ef7f-476b-a115-e9c734106efa service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:20.891609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:20.920043 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 868/1756] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:20.925787 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495167bb-ef7f-476b-a115-e9c734106efa service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:20.952510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495167bb-ef7f-476b-a115-e9c734106efa service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:20.971455 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418a50a-27c9-49fd-8fb7-0d5e01580803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:20.972457 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418a50a-27c9-49fd-8fb7-0d5e01580803 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:20.972652 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418a50a-27c9-49fd-8fb7-0d5e01580803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:20.973024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418a50a-27c9-49fd-8fb7-0d5e01580803 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:20.973847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418a50a-27c9-49fd-8fb7-0d5e01580803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:20.979811 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418a50a-27c9-49fd-8fb7-0d5e01580803 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:20.998429 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45472dc-69b9-4ec6-b26e-2e86d80eb36d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:21.005491 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b124a-3f79-4e3d-955e-b46a33b09048 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:21.006621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b124a-3f79-4e3d-955e-b46a33b09048 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:21.007119 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b124a-3f79-4e3d-955e-b46a33b09048 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:21.007744 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b124a-3f79-4e3d-955e-b46a33b09048 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:21.037592 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a64b124a-3f79-4e3d-955e-b46a33b09048 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:21.075293 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 869/1757] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:20 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:21.080287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418a50a-27c9-49fd-8fb7-0d5e01580803 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:21.109465 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418a50a-27c9-49fd-8fb7-0d5e01580803 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:21.124722 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f6292-301e-45f6-9efd-06b3c613b936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:21.126566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f6292-301e-45f6-9efd-06b3c613b936 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:21.126864 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f6292-301e-45f6-9efd-06b3c613b936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:21.127142 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f6292-301e-45f6-9efd-06b3c613b936 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:21.127856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f6292-301e-45f6-9efd-06b3c613b936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:21.141046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f6292-301e-45f6-9efd-06b3c613b936 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:21.198900 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 870/1758] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:21 2022] POST /identity/v3/users => generated 290 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:21.199799 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f6292-301e-45f6-9efd-06b3c613b936 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:21.225232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f6292-301e-45f6-9efd-06b3c613b936 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:21.240342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8928d-29fc-4b0b-8e40-60c4acc3a3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:21.241595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8928d-29fc-4b0b-8e40-60c4acc3a3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:21.241911 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8928d-29fc-4b0b-8e40-60c4acc3a3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:21.242241 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8928d-29fc-4b0b-8e40-60c4acc3a3d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:21.243364 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8928d-29fc-4b0b-8e40-60c4acc3a3d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:21.243759 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b124a-3f79-4e3d-955e-b46a33b09048 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:21.244990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8928d-29fc-4b0b-8e40-60c4acc3a3d2 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:21.246742 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b124a-3f79-4e3d-955e-b46a33b09048 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:21.274371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 889/1759] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:20 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:21.276563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b124a-3f79-4e3d-955e-b46a33b09048 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:21.291543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 871/1760] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:21.305482 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8928d-29fc-4b0b-8e40-60c4acc3a3d2 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:21.311857 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b124a-3f79-4e3d-955e-b46a33b09048 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:21.317385 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756defa-f0dc-4461-ac4e-dd1cddeabdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:21.318585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756defa-f0dc-4461-ac4e-dd1cddeabdc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:21.319064 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756defa-f0dc-4461-ac4e-dd1cddeabdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:21.319625 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756defa-f0dc-4461-ac4e-dd1cddeabdc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:21.320699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756defa-f0dc-4461-ac4e-dd1cddeabdc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:21.322568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756defa-f0dc-4461-ac4e-dd1cddeabdc7 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:21.341340 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 890/1761] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:21.342250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756defa-f0dc-4461-ac4e-dd1cddeabdc7 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:21.350489 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8928d-29fc-4b0b-8e40-60c4acc3a3d2 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:21.354576 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756defa-f0dc-4461-ac4e-dd1cddeabdc7 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:21.360215 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067281c-4830-40d7-9798-92b32b976e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:21.361554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067281c-4830-40d7-9798-92b32b976e7f admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:21.361933 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067281c-4830-40d7-9798-92b32b976e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:21.362310 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067281c-4830-40d7-9798-92b32b976e7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:21.363434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067281c-4830-40d7-9798-92b32b976e7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:21.365697 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067281c-4830-40d7-9798-92b32b976e7f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:21.372452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbeb26-084c-432a-a391-6bed6b9f8391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:21.373070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbeb26-084c-432a-a391-6bed6b9f8391 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:21.373363 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbeb26-084c-432a-a391-6bed6b9f8391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:21.373727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbeb26-084c-432a-a391-6bed6b9f8391 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:21.374470 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcbeb26-084c-432a-a391-6bed6b9f8391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:21.385601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcbeb26-084c-432a-a391-6bed6b9f8391 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:21.405084 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 891/1762] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:21.406630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067281c-4830-40d7-9798-92b32b976e7f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:21.437393 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 872/1763] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:21.438275 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbeb26-084c-432a-a391-6bed6b9f8391 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:21.441856 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067281c-4830-40d7-9798-92b32b976e7f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:21.455558 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2530f7-b564-454a-8469-7d725443fe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:21.456477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2530f7-b564-454a-8469-7d725443fe0a admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:21.456897 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2530f7-b564-454a-8469-7d725443fe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:21.457726 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2530f7-b564-454a-8469-7d725443fe0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:21.459083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2530f7-b564-454a-8469-7d725443fe0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:21.461052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2530f7-b564-454a-8469-7d725443fe0a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:21.467819 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbeb26-084c-432a-a391-6bed6b9f8391 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:21.481731 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 892/1764] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:21.483062 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2a6e2-35cb-4ea3-a6a6-a4d92a350212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:21.484207 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2a6e2-35cb-4ea3-a6a6-a4d92a350212 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:21.484527 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2a6e2-35cb-4ea3-a6a6-a4d92a350212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:21.484693 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2530f7-b564-454a-8469-7d725443fe0a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:21.484806 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2a6e2-35cb-4ea3-a6a6-a4d92a350212 admin admin] PATH_INFO: `/v3/projects/ca0907430f404357b91d31c57ff07b34/users/357bcb1f9da4403582ec77ec2ca27e70/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:21.498537 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2530f7-b564-454a-8469-7d725443fe0a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:21.504804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150249ee-6c66-4da0-b122-2dbc245deb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:21.505758 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150249ee-6c66-4da0-b122-2dbc245deb95 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:21.506264 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150249ee-6c66-4da0-b122-2dbc245deb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:21.506631 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150249ee-6c66-4da0-b122-2dbc245deb95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:21.507674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150249ee-6c66-4da0-b122-2dbc245deb95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:21.509413 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150249ee-6c66-4da0-b122-2dbc245deb95 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:21.526065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2a6e2-35cb-4ea3-a6a6-a4d92a350212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca0907430f404357b91d31c57ff07b34, user_id=357bcb1f9da4403582ec77ec2ca27e70, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:21.529818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 893/1765] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:21.530804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150249ee-6c66-4da0-b122-2dbc245deb95 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:21.533230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2a6e2-35cb-4ea3-a6a6-a4d92a350212 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:21.556807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150249ee-6c66-4da0-b122-2dbc245deb95 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:21.574227 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 873/1766] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:21 2022] PUT /identity/v3/projects/ca0907430f404357b91d31c57ff07b34/users/357bcb1f9da4403582ec77ec2ca27e70/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:21.575140 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2a6e2-35cb-4ea3-a6a6-a4d92a350212 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:21.661066 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:21.805524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e09b1-82e6-4684-bef2-d93766b19427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:21.806219 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2a6e2-35cb-4ea3-a6a6-a4d92a350212 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:21.808369 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e09b1-82e6-4684-bef2-d93766b19427 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:21.808716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e09b1-82e6-4684-bef2-d93766b19427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:21.808988 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e09b1-82e6-4684-bef2-d93766b19427 admin admin] PATH_INFO: `/v3/projects/48bcdec1c23940ec9e36fdc4b731116c/users/3b93e60aef8b474f8e104a9f4d99b479/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:21.818598 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64092a9b-69db-4e7c-9862-faab1fc9958b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:21.820909 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64092a9b-69db-4e7c-9862-faab1fc9958b admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:21.822131 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64092a9b-69db-4e7c-9862-faab1fc9958b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:21.824843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64092a9b-69db-4e7c-9862-faab1fc9958b admin admin] PATH_INFO: `/v3/projects/fd544156d6b140a4914c93d4a1c6ecb7/users/f656be1e69e84e7ea3679605349c63d8/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:21.857661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e09b1-82e6-4684-bef2-d93766b19427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48bcdec1c23940ec9e36fdc4b731116c, user_id=3b93e60aef8b474f8e104a9f4d99b479, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:21.860709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e09b1-82e6-4684-bef2-d93766b19427 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:21.892690 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64092a9b-69db-4e7c-9862-faab1fc9958b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd544156d6b140a4914c93d4a1c6ecb7, user_id=f656be1e69e84e7ea3679605349c63d8, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:21.893898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 894/1767] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:21 2022] PUT /identity/v3/projects/48bcdec1c23940ec9e36fdc4b731116c/users/3b93e60aef8b474f8e104a9f4d99b479/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:21.897396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0cdef-e1b9-480b-8875-303106b4f618 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:21.897961 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0cdef-e1b9-480b-8875-303106b4f618 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:21.899381 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64092a9b-69db-4e7c-9862-faab1fc9958b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:21.899811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0cdef-e1b9-480b-8875-303106b4f618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:21.932892 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 874/1768] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:21 2022] PUT /identity/v3/projects/fd544156d6b140a4914c93d4a1c6ecb7/users/f656be1e69e84e7ea3679605349c63d8/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:21.947111 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3444312-5eca-4c2b-8cfe-07123e4e0d6f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:21.947111 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3444312-5eca-4c2b-8cfe-07123e4e0d6f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:21.947631 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3444312-5eca-4c2b-8cfe-07123e4e0d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:21.955644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:21.983516 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-4dd0cdef-e1b9-480b-8875-303106b4f618 None None] MFA Rules not processed for user `357bcb1f9da4403582ec77ec2ca27e70`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:21.988427 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4dd0cdef-e1b9-480b-8875-303106b4f618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:22.023038 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f3444312-5eca-4c2b-8cfe-07123e4e0d6f None None] MFA Rules not processed for user `3b93e60aef8b474f8e104a9f4d99b479`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:22.027154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f3444312-5eca-4c2b-8cfe-07123e4e0d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:22.269846 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 895/1769] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:22.271525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f38747-c8e0-4ca4-a730-6b466ccd3540 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:22.271916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f38747-c8e0-4ca4-a730-6b466ccd3540 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:22.272133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f38747-c8e0-4ca4-a730-6b466ccd3540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:22.333828 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 875/1770] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:22.335423 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3444312-5eca-4c2b-8cfe-07123e4e0d6f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:22.339096 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f3444312-5eca-4c2b-8cfe-07123e4e0d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:22.341570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-94f38747-c8e0-4ca4-a730-6b466ccd3540 None None] MFA Rules not processed for user `f656be1e69e84e7ea3679605349c63d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:22.347092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-94f38747-c8e0-4ca4-a730-6b466ccd3540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:22.547555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 896/1771] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:22.550426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e176f41-a368-4d56-8671-e2db4984b364 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:22.552985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e176f41-a368-4d56-8671-e2db4984b364 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:22.553166 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e176f41-a368-4d56-8671-e2db4984b364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:22.606240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3444312-5eca-4c2b-8cfe-07123e4e0d6f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:22.611421 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91192efd-aea2-4159-a9e8-a0cac25ae92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:22.612302 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91192efd-aea2-4159-a9e8-a0cac25ae92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:22.612758 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91192efd-aea2-4159-a9e8-a0cac25ae92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:22.612928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91192efd-aea2-4159-a9e8-a0cac25ae92c admin admin] PATH_INFO: `/v3/projects/08002b21aa9d48dba64d381ac0ab01ea` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:22.614243 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91192efd-aea2-4159-a9e8-a0cac25ae92c admin admin] RBAC: Authorizing `identity:delete_project(project_id=08002b21aa9d48dba64d381ac0ab01ea)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:22.616703 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91192efd-aea2-4159-a9e8-a0cac25ae92c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:22.636848 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3e176f41-a368-4d56-8671-e2db4984b364 None None] MFA Rules not processed for user `19140330ae664e45b85f47f321496191`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:22.638120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3e176f41-a368-4d56-8671-e2db4984b364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:22.708201 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:22.749316 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-91192efd-aea2-4159-a9e8-a0cac25ae92c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08002b21aa9d48dba64d381ac0ab01ea', 'request_id': 'req-91192efd-aea2-4159-a9e8-a0cac25ae92c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:22.775220 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-91192efd-aea2-4159-a9e8-a0cac25ae92c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08002b21aa9d48dba64d381ac0ab01ea', 'request_id': 'req-91192efd-aea2-4159-a9e8-a0cac25ae92c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:22.801470 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-91192efd-aea2-4159-a9e8-a0cac25ae92c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08002b21aa9d48dba64d381ac0ab01ea', 'request_id': 'req-91192efd-aea2-4159-a9e8-a0cac25ae92c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:22.810476 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 897/1772] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:22.811338 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e176f41-a368-4d56-8671-e2db4984b364 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:22.824945 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e176f41-a368-4d56-8671-e2db4984b364 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:22.858862 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-91192efd-aea2-4159-a9e8-a0cac25ae92c admin admin] The token cache is being invalidate because project 08002b21aa9d48dba64d381ac0ab01ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:22.858862 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-91192efd-aea2-4159-a9e8-a0cac25ae92c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:22.858862 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-91192efd-aea2-4159-a9e8-a0cac25ae92c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:22.866303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 876/1773] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:22 2022] DELETE /identity/v3/projects/08002b21aa9d48dba64d381ac0ab01ea => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:22.866303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91192efd-aea2-4159-a9e8-a0cac25ae92c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:22.866303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-91192efd-aea2-4159-a9e8-a0cac25ae92c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:22.900301 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:22.999927 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08950465-27a3-4bed-b2d3-283bb460b0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:23.000654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08950465-27a3-4bed-b2d3-283bb460b0a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:23.000939 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08950465-27a3-4bed-b2d3-283bb460b0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:23.001214 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08950465-27a3-4bed-b2d3-283bb460b0a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:23.003504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08950465-27a3-4bed-b2d3-283bb460b0a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:23.007878 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08950465-27a3-4bed-b2d3-283bb460b0a6 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:23.059613 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91192efd-aea2-4159-a9e8-a0cac25ae92c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:23.066821 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 898/1774] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:22 2022] POST /identity/v3/projects => generated 373 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:23.082980 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08950465-27a3-4bed-b2d3-283bb460b0a6 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:23.084602 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397b764-b940-40be-a828-b47e033ffbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:23.085399 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397b764-b940-40be-a828-b47e033ffbbd admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:23.086422 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397b764-b940-40be-a828-b47e033ffbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:23.086823 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397b764-b940-40be-a828-b47e033ffbbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:23.087828 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397b764-b940-40be-a828-b47e033ffbbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:23.088146 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-08950465-27a3-4bed-b2d3-283bb460b0a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:23.090435 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397b764-b940-40be-a828-b47e033ffbbd admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:23.169123 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 877/1775] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:22 2022] POST /identity/v3/projects => generated 379 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:23.170140 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397b764-b940-40be-a828-b47e033ffbbd admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:23.172195 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b397b764-b940-40be-a828-b47e033ffbbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:23.236298 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08950465-27a3-4bed-b2d3-283bb460b0a6 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:23.355388 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397b764-b940-40be-a828-b47e033ffbbd admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:23.362909 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc4afd-1cf2-4f12-87ee-cc8f1e354e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:23.363889 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b34ff0-4738-4069-8b0e-dd2ef1828560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:23.365027 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc4afd-1cf2-4f12-87ee-cc8f1e354e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:23.365728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b34ff0-4738-4069-8b0e-dd2ef1828560 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:23.366451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc4afd-1cf2-4f12-87ee-cc8f1e354e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:23.366451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc4afd-1cf2-4f12-87ee-cc8f1e354e69 admin admin] PATH_INFO: `/v3/users/d8323c7522cd4868a9d2712db4344cd1` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:23.367256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b34ff0-4738-4069-8b0e-dd2ef1828560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:23.368511 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b34ff0-4738-4069-8b0e-dd2ef1828560 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:23.369430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc4afd-1cf2-4f12-87ee-cc8f1e354e69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8323c7522cd4868a9d2712db4344cd1)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:23.370230 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b34ff0-4738-4069-8b0e-dd2ef1828560 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:23.377011 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b34ff0-4738-4069-8b0e-dd2ef1828560 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:23.378049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc4afd-1cf2-4f12-87ee-cc8f1e354e69 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:23.438313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 899/1776] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:23 2022] POST /identity/v3/projects => generated 379 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:23.442859 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b34ff0-4738-4069-8b0e-dd2ef1828560 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:23.448660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-07b34ff0-4738-4069-8b0e-dd2ef1828560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:23.500589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-92bc4afd-1cf2-4f12-87ee-cc8f1e354e69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8323c7522cd4868a9d2712db4344cd1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:23.502183 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-92bc4afd-1cf2-4f12-87ee-cc8f1e354e69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8323c7522cd4868a9d2712db4344cd1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:23.510481 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:23.533970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-92bc4afd-1cf2-4f12-87ee-cc8f1e354e69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8323c7522cd4868a9d2712db4344cd1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:23.547147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-92bc4afd-1cf2-4f12-87ee-cc8f1e354e69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8323c7522cd4868a9d2712db4344cd1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:23.549161 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-92bc4afd-1cf2-4f12-87ee-cc8f1e354e69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8323c7522cd4868a9d2712db4344cd1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:23.628161 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 878/1777] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:23 2022] DELETE /identity/v3/users/d8323c7522cd4868a9d2712db4344cd1 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:23.629820 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60087d6d-8531-42d0-8505-80a4de5754aa None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:23.631379 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60087d6d-8531-42d0-8505-80a4de5754aa None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:23.632726 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60087d6d-8531-42d0-8505-80a4de5754aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:23.655124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:23.676263 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-60087d6d-8531-42d0-8505-80a4de5754aa None None] MFA Rules not processed for user `28b3e73439e64050b406a9f3e9b0160a`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:23.680007 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-60087d6d-8531-42d0-8505-80a4de5754aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:23.684818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b34ff0-4738-4069-8b0e-dd2ef1828560 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:23.824399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32326dd4-0204-41af-a467-6d9c52635f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:23.827590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 879/1778] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Jun 25 16:06:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:23.828784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32326dd4-0204-41af-a467-6d9c52635f44 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:23.829219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32326dd4-0204-41af-a467-6d9c52635f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:23.829608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32326dd4-0204-41af-a467-6d9c52635f44 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:23.830187 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60087d6d-8531-42d0-8505-80a4de5754aa None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:23.832267 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-60087d6d-8531-42d0-8505-80a4de5754aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:23.834763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32326dd4-0204-41af-a467-6d9c52635f44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:23.846262 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32326dd4-0204-41af-a467-6d9c52635f44 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:23.885984 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60087d6d-8531-42d0-8505-80a4de5754aa None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:23.891795 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352bd7e-6b24-46ee-b962-f54a68121061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:23.892856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352bd7e-6b24-46ee-b962-f54a68121061 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:23.893141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352bd7e-6b24-46ee-b962-f54a68121061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:23.893434 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352bd7e-6b24-46ee-b962-f54a68121061 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:23.894136 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352bd7e-6b24-46ee-b962-f54a68121061 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:23.896289 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352bd7e-6b24-46ee-b962-f54a68121061 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:23.926380 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 880/1779] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:23 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:23.939192 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352bd7e-6b24-46ee-b962-f54a68121061 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:23.943765 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d352bd7e-6b24-46ee-b962-f54a68121061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:23.949933 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 900/1780] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:23 2022] POST /identity/v3/users => generated 293 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:23.951102 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32326dd4-0204-41af-a467-6d9c52635f44 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:23.967750 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32326dd4-0204-41af-a467-6d9c52635f44 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:23.972691 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3ddcb-4141-452a-a9c1-639514b5f8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:23.973395 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3ddcb-4141-452a-a9c1-639514b5f8a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:23.973728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3ddcb-4141-452a-a9c1-639514b5f8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:23.974009 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3ddcb-4141-452a-a9c1-639514b5f8a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:23.975157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3ddcb-4141-452a-a9c1-639514b5f8a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:23.977128 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3ddcb-4141-452a-a9c1-639514b5f8a0 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:23.978797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352bd7e-6b24-46ee-b962-f54a68121061 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:23.990761 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6733b61-e83f-4679-aaa3-d7f5025dbb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:23.991451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6733b61-e83f-4679-aaa3-d7f5025dbb35 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:23.991730 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6733b61-e83f-4679-aaa3-d7f5025dbb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:23.993657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 901/1781] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:23.994555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3ddcb-4141-452a-a9c1-639514b5f8a0 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:23.994878 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6733b61-e83f-4679-aaa3-d7f5025dbb35 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:23.995817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6733b61-e83f-4679-aaa3-d7f5025dbb35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:23.996634 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-36b3ddcb-4141-452a-a9c1-639514b5f8a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:24.002413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6733b61-e83f-4679-aaa3-d7f5025dbb35 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:24.010631 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3ddcb-4141-452a-a9c1-639514b5f8a0 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:24.017093 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4e915-0c8d-49b0-83b0-bd1ae98800ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:24.017930 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4e915-0c8d-49b0-83b0-bd1ae98800ca admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:24.018221 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4e915-0c8d-49b0-83b0-bd1ae98800ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:24.018507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4e915-0c8d-49b0-83b0-bd1ae98800ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:24.020140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4e915-0c8d-49b0-83b0-bd1ae98800ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:24.022191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4e915-0c8d-49b0-83b0-bd1ae98800ca admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:24.039928 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 902/1782] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:24.048675 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4e915-0c8d-49b0-83b0-bd1ae98800ca admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:24.054642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 881/1783] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:23 2022] POST /identity/v3/users => generated 296 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:24.057472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6733b61-e83f-4679-aaa3-d7f5025dbb35 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:24.062370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4e915-0c8d-49b0-83b0-bd1ae98800ca admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:24.067987 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b5a22-5668-4563-905f-8992033c8d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:24.069363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b5a22-5668-4563-905f-8992033c8d2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:24.069651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b5a22-5668-4563-905f-8992033c8d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:24.069955 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b5a22-5668-4563-905f-8992033c8d2e admin admin] PATH_INFO: `/v3/projects/19ee5635cbee4829a7c96140bbda9437/users/dd1a9b7b4e1249b793cc8e79527ebf20/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:24.082895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6733b61-e83f-4679-aaa3-d7f5025dbb35 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:24.098152 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260f9a9-1ce6-4f8a-b53b-ff8544cedd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:24.100988 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260f9a9-1ce6-4f8a-b53b-ff8544cedd89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:24.100988 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260f9a9-1ce6-4f8a-b53b-ff8544cedd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:24.100988 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260f9a9-1ce6-4f8a-b53b-ff8544cedd89 admin admin] PATH_INFO: `/v3/projects/bf4435a5ccf848be9cd4b6e9dc1c269c/users/f668820f39d242d1bcb16b41163080cd/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:24.110093 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b5a22-5668-4563-905f-8992033c8d2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ee5635cbee4829a7c96140bbda9437, user_id=dd1a9b7b4e1249b793cc8e79527ebf20, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:24.113175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b5a22-5668-4563-905f-8992033c8d2e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:24.144344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260f9a9-1ce6-4f8a-b53b-ff8544cedd89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf4435a5ccf848be9cd4b6e9dc1c269c, user_id=f668820f39d242d1bcb16b41163080cd, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:24.145851 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 903/1784] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:24 2022] PUT /identity/v3/projects/19ee5635cbee4829a7c96140bbda9437/users/dd1a9b7b4e1249b793cc8e79527ebf20/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:24.147770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260f9a9-1ce6-4f8a-b53b-ff8544cedd89 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:24.151822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b5a22-5668-4563-905f-8992033c8d2e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:24.155828 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c60b5a22-5668-4563-905f-8992033c8d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:24.202019 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 882/1785] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:24 2022] PUT /identity/v3/projects/bf4435a5ccf848be9cd4b6e9dc1c269c/users/f668820f39d242d1bcb16b41163080cd/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:24.203007 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260f9a9-1ce6-4f8a-b53b-ff8544cedd89 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:24.402200 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260f9a9-1ce6-4f8a-b53b-ff8544cedd89 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:24.422561 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35fc30-a2d2-4735-ac4e-7f754ed638da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:24.423401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35fc30-a2d2-4735-ac4e-7f754ed638da admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:24.423750 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35fc30-a2d2-4735-ac4e-7f754ed638da admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:24.425088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35fc30-a2d2-4735-ac4e-7f754ed638da admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:24.426329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35fc30-a2d2-4735-ac4e-7f754ed638da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:24.440676 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35fc30-a2d2-4735-ac4e-7f754ed638da admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:24.458985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b5a22-5668-4563-905f-8992033c8d2e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:24.475169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 883/1786] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:24.475169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35fc30-a2d2-4735-ac4e-7f754ed638da admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:24.483518 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ec35fc30-a2d2-4735-ac4e-7f754ed638da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:24.531172 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35fc30-a2d2-4735-ac4e-7f754ed638da admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:24.561645 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf1b15-c29f-4801-8919-e506091f756a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:24.563345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf1b15-c29f-4801-8919-e506091f756a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:24.563807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf1b15-c29f-4801-8919-e506091f756a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:24.564267 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf1b15-c29f-4801-8919-e506091f756a admin admin] PATH_INFO: `/v3/projects/7d22394f24e84c45b5c1e696eadcf9f2` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:24.567418 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf1b15-c29f-4801-8919-e506091f756a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d22394f24e84c45b5c1e696eadcf9f2)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:24.569627 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf1b15-c29f-4801-8919-e506091f756a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:24.583749 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:DELETE=3,keystone:SELECT=90,keystone:INSERT=7 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:24.638344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:24.709689 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d677db-6136-432c-ad19-56adb1ae2a83 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19788c4239ca44f9a31a7d32086c8947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:24.710644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d677db-6136-432c-ad19-56adb1ae2a83 service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:24.711029 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d677db-6136-432c-ad19-56adb1ae2a83 service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:24.711423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d677db-6136-432c-ad19-56adb1ae2a83 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:24.757206 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-76bf1b15-c29f-4801-8919-e506091f756a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d22394f24e84c45b5c1e696eadcf9f2', 'request_id': 'req-76bf1b15-c29f-4801-8919-e506091f756a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:24.786146 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-76bf1b15-c29f-4801-8919-e506091f756a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d22394f24e84c45b5c1e696eadcf9f2', 'request_id': 'req-76bf1b15-c29f-4801-8919-e506091f756a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:24.798229 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-76bf1b15-c29f-4801-8919-e506091f756a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d22394f24e84c45b5c1e696eadcf9f2', 'request_id': 'req-76bf1b15-c29f-4801-8919-e506091f756a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:24.809868 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d677db-6136-432c-ad19-56adb1ae2a83 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:24.817566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d677db-6136-432c-ad19-56adb1ae2a83 service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:24.829490 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 904/1787] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Jun 25 16:06:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 25 16:06:24.831156 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d677db-6136-432c-ad19-56adb1ae2a83 service swift] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:24.839422 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-76bf1b15-c29f-4801-8919-e506091f756a admin admin] The token cache is being invalidate because project 7d22394f24e84c45b5c1e696eadcf9f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:24.839716 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-76bf1b15-c29f-4801-8919-e506091f756a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:24.841625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-76bf1b15-c29f-4801-8919-e506091f756a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:24.847347 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 884/1788] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:24 2022] DELETE /identity/v3/projects/7d22394f24e84c45b5c1e696eadcf9f2 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:24.848284 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf1b15-c29f-4801-8919-e506091f756a admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:24.857675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-76bf1b15-c29f-4801-8919-e506091f756a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:24.866634 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d677db-6136-432c-ad19-56adb1ae2a83 service swift] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:25.074494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91650a-80ab-4a71-8872-f4660f609dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:25.076069 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91650a-80ab-4a71-8872-f4660f609dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:25.076321 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91650a-80ab-4a71-8872-f4660f609dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:25.077248 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91650a-80ab-4a71-8872-f4660f609dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:25.078036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91650a-80ab-4a71-8872-f4660f609dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:25.082083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91650a-80ab-4a71-8872-f4660f609dc0 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:25.139789 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 905/1789] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:25.142975 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91650a-80ab-4a71-8872-f4660f609dc0 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:25.147130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-da91650a-80ab-4a71-8872-f4660f609dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:25.160547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf1b15-c29f-4801-8919-e506091f756a admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:25.190715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30e3d1-c6fc-498a-9c56-d9f7bf5e6427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:25.194517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30e3d1-c6fc-498a-9c56-d9f7bf5e6427 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:25.200906 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30e3d1-c6fc-498a-9c56-d9f7bf5e6427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:25.203396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30e3d1-c6fc-498a-9c56-d9f7bf5e6427 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:25.204376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30e3d1-c6fc-498a-9c56-d9f7bf5e6427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:25.206310 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30e3d1-c6fc-498a-9c56-d9f7bf5e6427 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:25.242149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91650a-80ab-4a71-8872-f4660f609dc0 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:25.246153 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 885/1790] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:25.248045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b5177-7838-4c6a-a09b-44a028c3852c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:25.256064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b5177-7838-4c6a-a09b-44a028c3852c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:25.256255 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b5177-7838-4c6a-a09b-44a028c3852c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:25.265409 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553dfc58-d3a8-47d2-acdc-eb6326e4ba4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:25.270518 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dfc58-d3a8-47d2-acdc-eb6326e4ba4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:25.271018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dfc58-d3a8-47d2-acdc-eb6326e4ba4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:25.275187 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dfc58-d3a8-47d2-acdc-eb6326e4ba4e admin admin] PATH_INFO: `/v3/projects/6752efaab1ca49cd94e6cc2e2deb5256/users/5b7f50c74f254ebcb7913c00483ade63/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:25.315256 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-266b5177-7838-4c6a-a09b-44a028c3852c None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:25.330169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-266b5177-7838-4c6a-a09b-44a028c3852c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:25.350022 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 886/1791] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:25.351673 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789de295-0d70-426b-9f85-930637a11062 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:25.353135 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789de295-0d70-426b-9f85-930637a11062 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:25.353436 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789de295-0d70-426b-9f85-930637a11062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:25.374659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553dfc58-d3a8-47d2-acdc-eb6326e4ba4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6752efaab1ca49cd94e6cc2e2deb5256, user_id=5b7f50c74f254ebcb7913c00483ade63, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:25.380057 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553dfc58-d3a8-47d2-acdc-eb6326e4ba4e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:25.417690 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-789de295-0d70-426b-9f85-930637a11062 None None] MFA Rules not processed for user `aaffb9f0bf714822829455c0487376ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:25.419237 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-789de295-0d70-426b-9f85-930637a11062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:25.438194 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 906/1792] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:25 2022] PUT /identity/v3/projects/6752efaab1ca49cd94e6cc2e2deb5256/users/5b7f50c74f254ebcb7913c00483ade63/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:25.439175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553dfc58-d3a8-47d2-acdc-eb6326e4ba4e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:25.450810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-553dfc58-d3a8-47d2-acdc-eb6326e4ba4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:25.566739 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 887/1793] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Jun 25 16:06:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:25.597486 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789de295-0d70-426b-9f85-930637a11062 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:25.769097 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:25.773843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:25.790121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553dfc58-d3a8-47d2-acdc-eb6326e4ba4e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:25.805863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15480795-f488-4432-a46d-4ea7c2da321a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:25.806882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15480795-f488-4432-a46d-4ea7c2da321a admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:25.807272 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15480795-f488-4432-a46d-4ea7c2da321a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:25.808353 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15480795-f488-4432-a46d-4ea7c2da321a admin admin] PATH_INFO: `/v3/projects/19ee5635cbee4829a7c96140bbda9437/users/dd1a9b7b4e1249b793cc8e79527ebf20/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:25.819948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15480795-f488-4432-a46d-4ea7c2da321a admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ee5635cbee4829a7c96140bbda9437, user_id=dd1a9b7b4e1249b793cc8e79527ebf20, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:25.850325 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15480795-f488-4432-a46d-4ea7c2da321a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:25.903151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 907/1794] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:25 2022] PUT /identity/v3/projects/19ee5635cbee4829a7c96140bbda9437/users/dd1a9b7b4e1249b793cc8e79527ebf20/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:25.905096 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829b93b-e006-4844-ae42-9ba1875e37fd None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:25.905642 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829b93b-e006-4844-ae42-9ba1875e37fd None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:25.906115 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829b93b-e006-4844-ae42-9ba1875e37fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:25.931099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789de295-0d70-426b-9f85-930637a11062 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:25.955734 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9829b93b-e006-4844-ae42-9ba1875e37fd None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:25.958327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9829b93b-e006-4844-ae42-9ba1875e37fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:26.110860 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 908/1795] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:26.110860 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829b93b-e006-4844-ae42-9ba1875e37fd None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:26.131287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4cc5bf-dcb9-4ad8-ad41-742d9a8925a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:26.135287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cc5bf-dcb9-4ad8-ad41-742d9a8925a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:26.137331 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cc5bf-dcb9-4ad8-ad41-742d9a8925a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:26.137887 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cc5bf-dcb9-4ad8-ad41-742d9a8925a3 admin admin] PATH_INFO: `/v3/projects/bf4435a5ccf848be9cd4b6e9dc1c269c/users/f668820f39d242d1bcb16b41163080cd/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:26.139460 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829b93b-e006-4844-ae42-9ba1875e37fd None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:26.144458 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4cc5bf-dcb9-4ad8-ad41-742d9a8925a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf4435a5ccf848be9cd4b6e9dc1c269c, user_id=f668820f39d242d1bcb16b41163080cd, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:26.149966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2e442-20d2-47c1-904e-59e04e69b965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:26.150744 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2e442-20d2-47c1-904e-59e04e69b965 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:26.151387 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2e442-20d2-47c1-904e-59e04e69b965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:26.151746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2e442-20d2-47c1-904e-59e04e69b965 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:26.152466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4cc5bf-dcb9-4ad8-ad41-742d9a8925a3 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:26.158233 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2e442-20d2-47c1-904e-59e04e69b965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:26.160229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde2e442-20d2-47c1-904e-59e04e69b965 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:26.202716 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 888/1796] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:25 2022] PUT /identity/v3/projects/bf4435a5ccf848be9cd4b6e9dc1c269c/users/f668820f39d242d1bcb16b41163080cd/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:26.202716 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4cc5bf-dcb9-4ad8-ad41-742d9a8925a3 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:26.203462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4e4cc5bf-dcb9-4ad8-ad41-742d9a8925a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:26.210261 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 909/1797] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:26.212980 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bc6e3-8288-4968-b780-0bce9cbb8654 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:26.213343 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bc6e3-8288-4968-b780-0bce9cbb8654 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:26.213740 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bc6e3-8288-4968-b780-0bce9cbb8654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:26.318695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-180bc6e3-8288-4968-b780-0bce9cbb8654 None None] MFA Rules not processed for user `dd1a9b7b4e1249b793cc8e79527ebf20`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:26.326873 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-180bc6e3-8288-4968-b780-0bce9cbb8654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:26.424601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4cc5bf-dcb9-4ad8-ad41-742d9a8925a3 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:26.436618 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81f454-d259-4285-bf74-b4a01f4f2215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:26.439305 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81f454-d259-4285-bf74-b4a01f4f2215 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:26.439616 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81f454-d259-4285-bf74-b4a01f4f2215 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:26.439877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81f454-d259-4285-bf74-b4a01f4f2215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:26.467479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81f454-d259-4285-bf74-b4a01f4f2215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:26.472909 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81f454-d259-4285-bf74-b4a01f4f2215 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:26.500483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 889/1798] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:26.501934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81f454-d259-4285-bf74-b4a01f4f2215 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:26.532481 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 910/1799] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:26.539412 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33462cfd-0886-40a3-bf0e-07c5b97fd9df None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:26.539903 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33462cfd-0886-40a3-bf0e-07c5b97fd9df None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:26.540146 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33462cfd-0886-40a3-bf0e-07c5b97fd9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:26.665950 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-33462cfd-0886-40a3-bf0e-07c5b97fd9df None None] MFA Rules not processed for user `f668820f39d242d1bcb16b41163080cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:26.667524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-33462cfd-0886-40a3-bf0e-07c5b97fd9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:26.675361 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81f454-d259-4285-bf74-b4a01f4f2215 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:26.692539 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8221e3-29a6-4144-8cc7-271d5f9e6492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:26.693238 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8221e3-29a6-4144-8cc7-271d5f9e6492 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:26.693510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8221e3-29a6-4144-8cc7-271d5f9e6492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:26.693783 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8221e3-29a6-4144-8cc7-271d5f9e6492 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:26.696298 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8221e3-29a6-4144-8cc7-271d5f9e6492 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:06:26.697123 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8221e3-29a6-4144-8cc7-271d5f9e6492 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:26.702827 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8221e3-29a6-4144-8cc7-271d5f9e6492 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:26.774847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 890/1800] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:06:26.776041 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8221e3-29a6-4144-8cc7-271d5f9e6492 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:26.787852 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8221e3-29a6-4144-8cc7-271d5f9e6492 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:26.806989 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75141b-1b2a-4ca6-a873-3a55824ae239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:26.808600 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75141b-1b2a-4ca6-a873-3a55824ae239 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:26.809114 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75141b-1b2a-4ca6-a873-3a55824ae239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:26.809822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75141b-1b2a-4ca6-a873-3a55824ae239 admin admin] PATH_INFO: `/v3/projects/6752efaab1ca49cd94e6cc2e2deb5256/users/5b7f50c74f254ebcb7913c00483ade63/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:26.819807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75141b-1b2a-4ca6-a873-3a55824ae239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6752efaab1ca49cd94e6cc2e2deb5256, user_id=5b7f50c74f254ebcb7913c00483ade63, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:26.827193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75141b-1b2a-4ca6-a873-3a55824ae239 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:26.832396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:26.873354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 891/1801] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:26 2022] PUT /identity/v3/projects/6752efaab1ca49cd94e6cc2e2deb5256/users/5b7f50c74f254ebcb7913c00483ade63/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:26.875745 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75141b-1b2a-4ca6-a873-3a55824ae239 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:26.894013 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6c75141b-1b2a-4ca6-a873-3a55824ae239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:26.903894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:26.954732 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 911/1802] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:26.956998 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33462cfd-0886-40a3-bf0e-07c5b97fd9df None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:26.982416 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33462cfd-0886-40a3-bf0e-07c5b97fd9df None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:27.091255 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75141b-1b2a-4ca6-a873-3a55824ae239 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:27.102507 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd2bc97-c323-40aa-a533-b222d418dcf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:27.105795 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2bc97-c323-40aa-a533-b222d418dcf8 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:27.106434 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2bc97-c323-40aa-a533-b222d418dcf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:27.107766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2bc97-c323-40aa-a533-b222d418dcf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:27.112486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743280c-de5d-4bee-867d-ed1412337418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:27.113396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743280c-de5d-4bee-867d-ed1412337418 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:27.113684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743280c-de5d-4bee-867d-ed1412337418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:27.133751 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743280c-de5d-4bee-867d-ed1412337418 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:27.134692 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3743280c-de5d-4bee-867d-ed1412337418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:27.141832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3743280c-de5d-4bee-867d-ed1412337418 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:27.223677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 912/1803] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:26 2022] POST /identity/v3/projects => generated 375 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:27.223677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e14177-fe75-4bd3-8fff-7a561ee8d9fd None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:27.223677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e14177-fe75-4bd3-8fff-7a561ee8d9fd None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:27.223677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e14177-fe75-4bd3-8fff-7a561ee8d9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:27.233846 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd2bc97-c323-40aa-a533-b222d418dcf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:27.235636 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd2bc97-c323-40aa-a533-b222d418dcf8 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:27.259551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e9e14177-fe75-4bd3-8fff-7a561ee8d9fd None None] MFA Rules not processed for user `5b7f50c74f254ebcb7913c00483ade63`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:27.260835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e9e14177-fe75-4bd3-8fff-7a561ee8d9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:27.293821 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 892/1804] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:27.294691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd2bc97-c323-40aa-a533-b222d418dcf8 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:27.297712 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3dd2bc97-c323-40aa-a533-b222d418dcf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:27.330362 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd2bc97-c323-40aa-a533-b222d418dcf8 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:27.469977 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 913/1805] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:27.476514 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e14177-fe75-4bd3-8fff-7a561ee8d9fd None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:27.496778 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da6c1e-30c6-4444-9781-d9a50a5e5d3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:27.497563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da6c1e-30c6-4444-9781-d9a50a5e5d3e service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:27.498110 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da6c1e-30c6-4444-9781-d9a50a5e5d3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:27.498434 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da6c1e-30c6-4444-9781-d9a50a5e5d3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:27.730727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:27.735522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da6c1e-30c6-4444-9781-d9a50a5e5d3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:27.737716 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da6c1e-30c6-4444-9781-d9a50a5e5d3e service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:27.744739 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e14177-fe75-4bd3-8fff-7a561ee8d9fd None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:27.772645 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 893/1806] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:27.773880 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da6c1e-30c6-4444-9781-d9a50a5e5d3e service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:27.786483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da6c1e-30c6-4444-9781-d9a50a5e5d3e service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:27.791429 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354a6db-1150-4241-914e-e28da62fe568 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:27.792217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354a6db-1150-4241-914e-e28da62fe568 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:27.792711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354a6db-1150-4241-914e-e28da62fe568 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:27.794226 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fe19d-c298-4b22-8bf8-4df02adc3bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:27.795165 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fe19d-c298-4b22-8bf8-4df02adc3bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:27.795503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fe19d-c298-4b22-8bf8-4df02adc3bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:27.795892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fe19d-c298-4b22-8bf8-4df02adc3bd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:27.806224 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354a6db-1150-4241-914e-e28da62fe568 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:27.806560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3fe19d-c298-4b22-8bf8-4df02adc3bd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:27.813402 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3fe19d-c298-4b22-8bf8-4df02adc3bd1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:27.822722 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354a6db-1150-4241-914e-e28da62fe568 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:27.824850 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354a6db-1150-4241-914e-e28da62fe568 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:27.855975 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:27.882379 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 894/1807] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:27.883797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354a6db-1150-4241-914e-e28da62fe568 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:27.885706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9354a6db-1150-4241-914e-e28da62fe568 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:27.945997 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 914/1808] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:27 2022] POST /identity/v3/users => generated 288 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:27.947076 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fe19d-c298-4b22-8bf8-4df02adc3bd1 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:27.950032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354a6db-1150-4241-914e-e28da62fe568 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:27.975018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996848b2-4c9e-495c-ae3c-ee44487f74f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:27.980435 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996848b2-4c9e-495c-ae3c-ee44487f74f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:27.982821 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996848b2-4c9e-495c-ae3c-ee44487f74f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:27.984311 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996848b2-4c9e-495c-ae3c-ee44487f74f2 admin admin] PATH_INFO: `/v3/users/bd56b922b0e14da1a299b5052cd8bc95` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:27.989382 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996848b2-4c9e-495c-ae3c-ee44487f74f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd56b922b0e14da1a299b5052cd8bc95)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:27.997932 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fe19d-c298-4b22-8bf8-4df02adc3bd1 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:28.017914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50a4fa-cb78-4752-bd03-c5a92140cc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:28.019298 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50a4fa-cb78-4752-bd03-c5a92140cc20 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:28.019966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50a4fa-cb78-4752-bd03-c5a92140cc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:28.021926 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50a4fa-cb78-4752-bd03-c5a92140cc20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:28.023256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50a4fa-cb78-4752-bd03-c5a92140cc20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:28.024619 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996848b2-4c9e-495c-ae3c-ee44487f74f2 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:28.026850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50a4fa-cb78-4752-bd03-c5a92140cc20 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:28.070757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 915/1809] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:28.072045 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50a4fa-cb78-4752-bd03-c5a92140cc20 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:28.119325 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50a4fa-cb78-4752-bd03-c5a92140cc20 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:28.142272 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150de58-b359-4e2e-b6d9-889fdbd7bb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:28.142969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150de58-b359-4e2e-b6d9-889fdbd7bb97 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:28.143249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150de58-b359-4e2e-b6d9-889fdbd7bb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:28.143659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150de58-b359-4e2e-b6d9-889fdbd7bb97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:28.151034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150de58-b359-4e2e-b6d9-889fdbd7bb97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:28.153054 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150de58-b359-4e2e-b6d9-889fdbd7bb97 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:28.220170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 916/1810] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:28.226118 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150de58-b359-4e2e-b6d9-889fdbd7bb97 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:28.245714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-996848b2-4c9e-495c-ae3c-ee44487f74f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd56b922b0e14da1a299b5052cd8bc95'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:28.250641 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-996848b2-4c9e-495c-ae3c-ee44487f74f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd56b922b0e14da1a299b5052cd8bc95'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:28.253558 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150de58-b359-4e2e-b6d9-889fdbd7bb97 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:28.269244 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd87ed4-3a17-425c-b828-12185089b90c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:28.270386 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd87ed4-3a17-425c-b828-12185089b90c service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:28.270861 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd87ed4-3a17-425c-b828-12185089b90c service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:28.271408 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd87ed4-3a17-425c-b828-12185089b90c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:28.276246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9fd87ed4-3a17-425c-b828-12185089b90c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:28.283197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-996848b2-4c9e-495c-ae3c-ee44487f74f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd56b922b0e14da1a299b5052cd8bc95'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:28.318470 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-996848b2-4c9e-495c-ae3c-ee44487f74f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd56b922b0e14da1a299b5052cd8bc95'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:28.319379 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-996848b2-4c9e-495c-ae3c-ee44487f74f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd56b922b0e14da1a299b5052cd8bc95'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:28.390452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 895/1811] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:27 2022] DELETE /identity/v3/users/bd56b922b0e14da1a299b5052cd8bc95 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:28.396718 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996848b2-4c9e-495c-ae3c-ee44487f74f2 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:28.401579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-996848b2-4c9e-495c-ae3c-ee44487f74f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:28.470792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd87ed4-3a17-425c-b828-12185089b90c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:28.478412 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd87ed4-3a17-425c-b828-12185089b90c service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:28.525712 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 917/1812] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:28.531365 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdc816-69c2-450e-bbf3-ed1e18cb8c78 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:28.531798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdc816-69c2-450e-bbf3-ed1e18cb8c78 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:28.532102 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdc816-69c2-450e-bbf3-ed1e18cb8c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:28.597533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996848b2-4c9e-495c-ae3c-ee44487f74f2 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:28.616634 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241454e-8380-4223-84fb-eed4ea8ac99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:28.617481 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f4bdc816-69c2-450e-bbf3-ed1e18cb8c78 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:28.618518 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241454e-8380-4223-84fb-eed4ea8ac99e admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:28.619127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241454e-8380-4223-84fb-eed4ea8ac99e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:28.619624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241454e-8380-4223-84fb-eed4ea8ac99e admin admin] PATH_INFO: `/v3/projects/41cf47caff97420390fea1a6ed4ed6da/users/8db0221ba50f4497b65bafdf37b9672b/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:28.623289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f4bdc816-69c2-450e-bbf3-ed1e18cb8c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:28.640028 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241454e-8380-4223-84fb-eed4ea8ac99e admin admin] RBAC: Authorizing `identity:create_grant(project_id=41cf47caff97420390fea1a6ed4ed6da, user_id=8db0221ba50f4497b65bafdf37b9672b, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:28.643986 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241454e-8380-4223-84fb-eed4ea8ac99e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:28.653948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 918/1813] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:28.658454 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdc816-69c2-450e-bbf3-ed1e18cb8c78 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:28.662861 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f4bdc816-69c2-450e-bbf3-ed1e18cb8c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:28.690152 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 896/1814] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:28 2022] PUT /identity/v3/projects/41cf47caff97420390fea1a6ed4ed6da/users/8db0221ba50f4497b65bafdf37b9672b/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:28.691928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529ca61-e60a-442a-a5f5-4c278f54d09d None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:28.692518 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529ca61-e60a-442a-a5f5-4c278f54d09d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:28.693277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529ca61-e60a-442a-a5f5-4c278f54d09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:28.767534 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a529ca61-e60a-442a-a5f5-4c278f54d09d None None] MFA Rules not processed for user `8db0221ba50f4497b65bafdf37b9672b`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:28.769315 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a529ca61-e60a-442a-a5f5-4c278f54d09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:28.828665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:28.847929 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdc816-69c2-450e-bbf3-ed1e18cb8c78 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:28.904390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 897/1815] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:28.906071 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529ca61-e60a-442a-a5f5-4c278f54d09d None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:28.914555 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a529ca61-e60a-442a-a5f5-4c278f54d09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:29.008209 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529ca61-e60a-442a-a5f5-4c278f54d09d None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:29.013136 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568414b-fba4-4a6a-b303-997a6848ac34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:29.013330 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec2b68-b7fc-4a96-853b-29257269adfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:29.013947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec2b68-b7fc-4a96-853b-29257269adfc service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:29.014050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568414b-fba4-4a6a-b303-997a6848ac34 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:29.014234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec2b68-b7fc-4a96-853b-29257269adfc service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:29.014408 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568414b-fba4-4a6a-b303-997a6848ac34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:29.014507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec2b68-b7fc-4a96-853b-29257269adfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:29.014856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568414b-fba4-4a6a-b303-997a6848ac34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:29.024868 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-15ec2b68-b7fc-4a96-853b-29257269adfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:29.103466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568414b-fba4-4a6a-b303-997a6848ac34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:29.105387 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568414b-fba4-4a6a-b303-997a6848ac34 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:29.118549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 898/1816] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:29.119685 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568414b-fba4-4a6a-b303-997a6848ac34 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:29.122714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:29.138564 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568414b-fba4-4a6a-b303-997a6848ac34 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:29.145904 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14776b42-627b-4fc9-be73-242608c0569b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:29.146657 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14776b42-627b-4fc9-be73-242608c0569b admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:29.146937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14776b42-627b-4fc9-be73-242608c0569b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:29.147505 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14776b42-627b-4fc9-be73-242608c0569b admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:29.149459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14776b42-627b-4fc9-be73-242608c0569b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:29.152798 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14776b42-627b-4fc9-be73-242608c0569b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:29.194016 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec2b68-b7fc-4a96-853b-29257269adfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:29.196783 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec2b68-b7fc-4a96-853b-29257269adfc service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:29.214760 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 899/1817] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:29 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:29.217223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14776b42-627b-4fc9-be73-242608c0569b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:29.220723 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 919/1818] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:29.221584 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec2b68-b7fc-4a96-853b-29257269adfc service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:29.238831 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-14776b42-627b-4fc9-be73-242608c0569b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:29.249140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec2b68-b7fc-4a96-853b-29257269adfc service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:29.439761 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc403e4-c0cc-46ce-a72b-cbeb766831cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:29.440496 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc403e4-c0cc-46ce-a72b-cbeb766831cc admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:29.440773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc403e4-c0cc-46ce-a72b-cbeb766831cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:29.441164 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc403e4-c0cc-46ce-a72b-cbeb766831cc admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:29.441876 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc403e4-c0cc-46ce-a72b-cbeb766831cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:29.448887 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc403e4-c0cc-46ce-a72b-cbeb766831cc admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:29.465807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:29.482097 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14776b42-627b-4fc9-be73-242608c0569b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:29.499493 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecab61-8e59-4c33-929b-98d493bbb459 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1a9b7b4e1249b793cc8e79527ebf20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ee5635cbee4829a7c96140bbda9437', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:29.501353 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecab61-8e59-4c33-929b-98d493bbb459 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:29.501634 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecab61-8e59-4c33-929b-98d493bbb459 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:29.501918 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecab61-8e59-4c33-929b-98d493bbb459 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:29.507824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecab61-8e59-4c33-929b-98d493bbb459 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:06:29.508377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecab61-8e59-4c33-929b-98d493bbb459 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:29.510063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecab61-8e59-4c33-929b-98d493bbb459 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:29.524799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 920/1819] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:29 2022] POST /identity/v3/users => generated 295 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:29.525859 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc403e4-c0cc-46ce-a72b-cbeb766831cc admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:29.549322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc403e4-c0cc-46ce-a72b-cbeb766831cc admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:29.552948 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 900/1820] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:06:29.555651 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecab61-8e59-4c33-929b-98d493bbb459 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:29.566928 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedff789-70e9-42e4-bafc-9357ac72ebdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:29.567744 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedff789-70e9-42e4-bafc-9357ac72ebdc admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:29.568036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedff789-70e9-42e4-bafc-9357ac72ebdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:29.568323 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedff789-70e9-42e4-bafc-9357ac72ebdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:29.569158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedff789-70e9-42e4-bafc-9357ac72ebdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:29.574717 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecab61-8e59-4c33-929b-98d493bbb459 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:29.577550 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedff789-70e9-42e4-bafc-9357ac72ebdc admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:29.581387 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5102a-982a-4406-b27c-2274a27f6b59 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1a9b7b4e1249b793cc8e79527ebf20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ee5635cbee4829a7c96140bbda9437', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:29.582146 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5102a-982a-4406-b27c-2274a27f6b59 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:29.582752 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5102a-982a-4406-b27c-2274a27f6b59 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:29.582964 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5102a-982a-4406-b27c-2274a27f6b59 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:29.584298 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e5102a-982a-4406-b27c-2274a27f6b59 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:29.587279 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e5102a-982a-4406-b27c-2274a27f6b59 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:29.599590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 921/1821] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:29.609370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedff789-70e9-42e4-bafc-9357ac72ebdc admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:29.641483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedff789-70e9-42e4-bafc-9357ac72ebdc admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:29.671400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe371ed-ed8d-47d0-b14e-536269d990f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:29.676138 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe371ed-ed8d-47d0-b14e-536269d990f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:29.676792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe371ed-ed8d-47d0-b14e-536269d990f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:29.678376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe371ed-ed8d-47d0-b14e-536269d990f9 admin admin] PATH_INFO: `/v3/projects/6cb8433b8306413c91268b71420d873c/users/0cd971a0002f417b86c29d33ff3efa41/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:29.721455 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 901/1822] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:29 2022] POST /identity/v3/projects => generated 354 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:06:29.722881 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5102a-982a-4406-b27c-2274a27f6b59 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:29.740261 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe371ed-ed8d-47d0-b14e-536269d990f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cb8433b8306413c91268b71420d873c, user_id=0cd971a0002f417b86c29d33ff3efa41, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:29.743773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe371ed-ed8d-47d0-b14e-536269d990f9 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:29.778793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 922/1823] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:29 2022] PUT /identity/v3/projects/6cb8433b8306413c91268b71420d873c/users/0cd971a0002f417b86c29d33ff3efa41/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:29.781778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe371ed-ed8d-47d0-b14e-536269d990f9 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:29.990139 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5102a-982a-4406-b27c-2274a27f6b59 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:30.051724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe371ed-ed8d-47d0-b14e-536269d990f9 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:30.073645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62145002-98f1-445a-8eb3-fd2e0ca13a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:30.078023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62145002-98f1-445a-8eb3-fd2e0ca13a65 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:30.078401 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62145002-98f1-445a-8eb3-fd2e0ca13a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:30.079924 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62145002-98f1-445a-8eb3-fd2e0ca13a65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:30.080902 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62145002-98f1-445a-8eb3-fd2e0ca13a65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:30.087041 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62145002-98f1-445a-8eb3-fd2e0ca13a65 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:30.126956 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 923/1824] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:30.128141 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62145002-98f1-445a-8eb3-fd2e0ca13a65 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:30.157371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62145002-98f1-445a-8eb3-fd2e0ca13a65 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:30.201354 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e027037-4a36-4fc9-8f89-4059bb45b7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:30.202377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e027037-4a36-4fc9-8f89-4059bb45b7c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:30.202720 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e027037-4a36-4fc9-8f89-4059bb45b7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:30.203044 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e027037-4a36-4fc9-8f89-4059bb45b7c1 admin admin] PATH_INFO: `/v3/projects/6cb8433b8306413c91268b71420d873c/users/0cd971a0002f417b86c29d33ff3efa41/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:30.207402 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e027037-4a36-4fc9-8f89-4059bb45b7c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cb8433b8306413c91268b71420d873c, user_id=0cd971a0002f417b86c29d33ff3efa41, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:30.210906 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17365b68-c84a-4792-850a-81c118d92cf0 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1a9b7b4e1249b793cc8e79527ebf20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ee5635cbee4829a7c96140bbda9437', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:30.211902 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e027037-4a36-4fc9-8f89-4059bb45b7c1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:30.212345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17365b68-c84a-4792-850a-81c118d92cf0 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:30.212820 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17365b68-c84a-4792-850a-81c118d92cf0 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:30.213328 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17365b68-c84a-4792-850a-81c118d92cf0 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:30.214698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17365b68-c84a-4792-850a-81c118d92cf0 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:06:30.215314 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17365b68-c84a-4792-850a-81c118d92cf0 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:30.219508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17365b68-c84a-4792-850a-81c118d92cf0 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:30.228306 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 924/1825] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:30 2022] PUT /identity/v3/projects/6cb8433b8306413c91268b71420d873c/users/0cd971a0002f417b86c29d33ff3efa41/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:30.244218 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9a0ec-1c39-4955-8a50-649bd25ea9d0 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:30.244619 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9a0ec-1c39-4955-8a50-649bd25ea9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:30.244944 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9a0ec-1c39-4955-8a50-649bd25ea9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:30.249134 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 902/1826] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 527 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 16:06:30.300352 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:30.322979 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17365b68-c84a-4792-850a-81c118d92cf0 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:30.323777 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-4de9a0ec-1c39-4955-8a50-649bd25ea9d0 None None] MFA Rules not processed for user `0cd971a0002f417b86c29d33ff3efa41`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:30.325410 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4de9a0ec-1c39-4955-8a50-649bd25ea9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:30.338883 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17365b68-c84a-4792-850a-81c118d92cf0 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:30.503614 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 925/1827] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:30.521575 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9a0ec-1c39-4955-8a50-649bd25ea9d0 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:30.560176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1e0dd-bbfa-41f3-b977-e10e3bc2f473 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1a9b7b4e1249b793cc8e79527ebf20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ee5635cbee4829a7c96140bbda9437', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:30.563785 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1e0dd-bbfa-41f3-b977-e10e3bc2f473 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:30.564114 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1e0dd-bbfa-41f3-b977-e10e3bc2f473 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:30.564485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1e0dd-bbfa-41f3-b977-e10e3bc2f473 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] PATH_INFO: `/v3/projects/3da7e04ee9854a5b99de1c04d6142134` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:30.581622 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1e0dd-bbfa-41f3-b977-e10e3bc2f473 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3da7e04ee9854a5b99de1c04d6142134)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:30.584176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1e0dd-bbfa-41f3-b977-e10e3bc2f473 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:30.612487 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:30.688582 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8ca1e0dd-bbfa-41f3-b977-e10e3bc2f473 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3da7e04ee9854a5b99de1c04d6142134', 'request_id': 'req-8ca1e0dd-bbfa-41f3-b977-e10e3bc2f473'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:30.715655 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8ca1e0dd-bbfa-41f3-b977-e10e3bc2f473 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3da7e04ee9854a5b99de1c04d6142134', 'request_id': 'req-8ca1e0dd-bbfa-41f3-b977-e10e3bc2f473'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:30.717888 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9a0ec-1c39-4955-8a50-649bd25ea9d0 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:30.726892 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8ca1e0dd-bbfa-41f3-b977-e10e3bc2f473 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3da7e04ee9854a5b99de1c04d6142134', 'request_id': 'req-8ca1e0dd-bbfa-41f3-b977-e10e3bc2f473'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:30.763264 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8ca1e0dd-bbfa-41f3-b977-e10e3bc2f473 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] The token cache is being invalidate because project 3da7e04ee9854a5b99de1c04d6142134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:30.763264 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8ca1e0dd-bbfa-41f3-b977-e10e3bc2f473 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:30.763264 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8ca1e0dd-bbfa-41f3-b977-e10e3bc2f473 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:30.763646 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 903/1828] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:30 2022] DELETE /identity/v3/projects/3da7e04ee9854a5b99de1c04d6142134 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:30.934988 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880a59c-07d9-4be8-823b-dc88a06ff6ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:30.935775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880a59c-07d9-4be8-823b-dc88a06ff6ed service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:30.936092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880a59c-07d9-4be8-823b-dc88a06ff6ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:30.936384 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880a59c-07d9-4be8-823b-dc88a06ff6ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:30.940124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1880a59c-07d9-4be8-823b-dc88a06ff6ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:30.983095 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1e0dd-bbfa-41f3-b977-e10e3bc2f473 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:30.985055 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8ca1e0dd-bbfa-41f3-b977-e10e3bc2f473 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:31.171955 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880a59c-07d9-4be8-823b-dc88a06ff6ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:31.172765 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1e0dd-bbfa-41f3-b977-e10e3bc2f473 tempest-VolumeQuotaClassesTest-770611972 tempest-VolumeQuotaClassesTest-770611972-project-admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:31.183443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d033e-6714-41fb-b65e-5bffb277de29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:31.184563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d033e-6714-41fb-b65e-5bffb277de29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:31.184895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d033e-6714-41fb-b65e-5bffb277de29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:31.185268 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d033e-6714-41fb-b65e-5bffb277de29 admin admin] PATH_INFO: `/v3/projects/6fc72eafb8684a9dab826b0c2e7f3847` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:31.193540 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d033e-6714-41fb-b65e-5bffb277de29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fc72eafb8684a9dab826b0c2e7f3847)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:31.196075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d033e-6714-41fb-b65e-5bffb277de29 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:31.204498 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880a59c-07d9-4be8-823b-dc88a06ff6ed service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:31.262535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 926/1829] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 743 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:31.404278 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c20d033e-6714-41fb-b65e-5bffb277de29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fc72eafb8684a9dab826b0c2e7f3847', 'request_id': 'req-c20d033e-6714-41fb-b65e-5bffb277de29'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:31.437378 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c20d033e-6714-41fb-b65e-5bffb277de29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fc72eafb8684a9dab826b0c2e7f3847', 'request_id': 'req-c20d033e-6714-41fb-b65e-5bffb277de29'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:31.459390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c20d033e-6714-41fb-b65e-5bffb277de29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fc72eafb8684a9dab826b0c2e7f3847', 'request_id': 'req-c20d033e-6714-41fb-b65e-5bffb277de29'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:31.502266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c20d033e-6714-41fb-b65e-5bffb277de29 admin admin] The token cache is being invalidate because project 6fc72eafb8684a9dab826b0c2e7f3847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:31.505220 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c20d033e-6714-41fb-b65e-5bffb277de29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:31.508149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c20d033e-6714-41fb-b65e-5bffb277de29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:31.512891 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 904/1830] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:30 2022] DELETE /identity/v3/projects/6fc72eafb8684a9dab826b0c2e7f3847 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:31.517127 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880a59c-07d9-4be8-823b-dc88a06ff6ed service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:31.528293 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:31.543035 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880a59c-07d9-4be8-823b-dc88a06ff6ed service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:31.740996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40d536-dff0-4227-a9fe-d25d2ca660a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:31.742140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40d536-dff0-4227-a9fe-d25d2ca660a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:31.742517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40d536-dff0-4227-a9fe-d25d2ca660a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:31.742968 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40d536-dff0-4227-a9fe-d25d2ca660a8 admin admin] PATH_INFO: `/v3/users/a401d11c9ae446cc9418fa631e7dbe58` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:31.745166 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40d536-dff0-4227-a9fe-d25d2ca660a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a401d11c9ae446cc9418fa631e7dbe58)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:31.756755 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40d536-dff0-4227-a9fe-d25d2ca660a8 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:31.889690 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6f40d536-dff0-4227-a9fe-d25d2ca660a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a401d11c9ae446cc9418fa631e7dbe58'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:31.911011 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6f40d536-dff0-4227-a9fe-d25d2ca660a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a401d11c9ae446cc9418fa631e7dbe58'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:31.912377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6f40d536-dff0-4227-a9fe-d25d2ca660a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a401d11c9ae446cc9418fa631e7dbe58'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:31.928659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6f40d536-dff0-4227-a9fe-d25d2ca660a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a401d11c9ae446cc9418fa631e7dbe58'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:31.930090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6f40d536-dff0-4227-a9fe-d25d2ca660a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a401d11c9ae446cc9418fa631e7dbe58'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:31.987816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 927/1831] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:31 2022] DELETE /identity/v3/users/a401d11c9ae446cc9418fa631e7dbe58 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:32.287988 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d033e-6714-41fb-b65e-5bffb277de29 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:32.290378 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c20d033e-6714-41fb-b65e-5bffb277de29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:32.311996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec9de6-ff77-4c18-9b93-741298ad5d1d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:32.312671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec9de6-ff77-4c18-9b93-741298ad5d1d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:32.313134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec9de6-ff77-4c18-9b93-741298ad5d1d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:32.315308 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 928/1832] 10.222.0.13 () {54 vars in 921 bytes} [Sat Jun 25 16:06:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:06:32.328812 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d033e-6714-41fb-b65e-5bffb277de29 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:32.395268 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:32.416576 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affad5f0-d380-423f-bee9-042b86517878 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:32.416975 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affad5f0-d380-423f-bee9-042b86517878 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:32.417371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affad5f0-d380-423f-bee9-042b86517878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:32.544117 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-affad5f0-d380-423f-bee9-042b86517878 None None] MFA Rules not processed for user `3622c5ccb7534e7c8e8342b4f6643bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:32.554333 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-affad5f0-d380-423f-bee9-042b86517878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:32.656350 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a5125-8efb-4d90-928a-2ad63b48edf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:32.658485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a5125-8efb-4d90-928a-2ad63b48edf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:32.658768 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a5125-8efb-4d90-928a-2ad63b48edf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:32.660187 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a5125-8efb-4d90-928a-2ad63b48edf0 admin admin] PATH_INFO: `/v3/users/3b93e60aef8b474f8e104a9f4d99b479` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:32.672311 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a5125-8efb-4d90-928a-2ad63b48edf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b93e60aef8b474f8e104a9f4d99b479)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:32.684880 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a5125-8efb-4d90-928a-2ad63b48edf0 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:32.858625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-953a5125-8efb-4d90-928a-2ad63b48edf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b93e60aef8b474f8e104a9f4d99b479'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:32.863366 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-953a5125-8efb-4d90-928a-2ad63b48edf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b93e60aef8b474f8e104a9f4d99b479'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:32.881370 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-953a5125-8efb-4d90-928a-2ad63b48edf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b93e60aef8b474f8e104a9f4d99b479'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:32.894970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 929/1833] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Jun 25 16:06:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:06:32.898213 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-953a5125-8efb-4d90-928a-2ad63b48edf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b93e60aef8b474f8e104a9f4d99b479'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:32.900506 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-953a5125-8efb-4d90-928a-2ad63b48edf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b93e60aef8b474f8e104a9f4d99b479'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:32.901837 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88656e19-2653-45be-9c68-0b2dc02ba494 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:32.902391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88656e19-2653-45be-9c68-0b2dc02ba494 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:32.902754 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88656e19-2653-45be-9c68-0b2dc02ba494 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:32.905870 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 930/1834] 10.222.0.13 () {54 vars in 921 bytes} [Sat Jun 25 16:06:32 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:06:32.956026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 905/1835] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:32 2022] DELETE /identity/v3/users/3b93e60aef8b474f8e104a9f4d99b479 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:06:32.958311 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d6097-3ebd-40ea-b8e4-4027466c44c5 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:32.959571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d6097-3ebd-40ea-b8e4-4027466c44c5 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:32.960057 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d6097-3ebd-40ea-b8e4-4027466c44c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:33.024900 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b52d6097-3ebd-40ea-b8e4-4027466c44c5 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:33.026266 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b52d6097-3ebd-40ea-b8e4-4027466c44c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:33.026639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216411b-bca9-4525-80c6-e495e6d20bbb None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:33.027039 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216411b-bca9-4525-80c6-e495e6d20bbb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:33.028851 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216411b-bca9-4525-80c6-e495e6d20bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:33.067769 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:33.071188 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-1216411b-bca9-4525-80c6-e495e6d20bbb None None] MFA Rules not processed for user `3622c5ccb7534e7c8e8342b4f6643bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:33.073108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1216411b-bca9-4525-80c6-e495e6d20bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:33.198775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 931/1836] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:33.249321 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d6097-3ebd-40ea-b8e4-4027466c44c5 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:33.251151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b52d6097-3ebd-40ea-b8e4-4027466c44c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:33.279085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 906/1837] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Jun 25 16:06:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:06:33.431568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d6097-3ebd-40ea-b8e4-4027466c44c5 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:33.436514 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a6765-f626-4a77-a1a2-b186009555f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:33.437354 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a6765-f626-4a77-a1a2-b186009555f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:33.437779 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a6765-f626-4a77-a1a2-b186009555f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:33.438160 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a6765-f626-4a77-a1a2-b186009555f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:33.465367 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a6765-f626-4a77-a1a2-b186009555f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:33.467409 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a6765-f626-4a77-a1a2-b186009555f1 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:33.490235 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 932/1838] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:06:33.559286 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216411b-bca9-4525-80c6-e495e6d20bbb None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:33.561652 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1216411b-bca9-4525-80c6-e495e6d20bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:33.602806 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216411b-bca9-4525-80c6-e495e6d20bbb None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:33.620180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc62c12-ad2b-4fc1-bc4e-c420cdce16bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:33.621027 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc62c12-ad2b-4fc1-bc4e-c420cdce16bf service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:33.621374 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc62c12-ad2b-4fc1-bc4e-c420cdce16bf service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:33.621658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc62c12-ad2b-4fc1-bc4e-c420cdce16bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:33.660307 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc62c12-ad2b-4fc1-bc4e-c420cdce16bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:33.662644 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc62c12-ad2b-4fc1-bc4e-c420cdce16bf service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:33.686587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 907/1839] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:33.761192 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a6765-f626-4a77-a1a2-b186009555f1 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:33.762547 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-406a6765-f626-4a77-a1a2-b186009555f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:33.786758 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a6765-f626-4a77-a1a2-b186009555f1 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:33.800764 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6357fe7-fa0e-486c-b24a-8b4da6889501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:33.805504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6357fe7-fa0e-486c-b24a-8b4da6889501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:33.805874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6357fe7-fa0e-486c-b24a-8b4da6889501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:33.806151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6357fe7-fa0e-486c-b24a-8b4da6889501 admin admin] PATH_INFO: `/v3/projects/fc864821e72d4825943e0d28ee3e1b1c` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:33.808965 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6357fe7-fa0e-486c-b24a-8b4da6889501 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc864821e72d4825943e0d28ee3e1b1c)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:33.816536 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6357fe7-fa0e-486c-b24a-8b4da6889501 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:33.903803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc62c12-ad2b-4fc1-bc4e-c420cdce16bf service nova] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:33.923225 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc62c12-ad2b-4fc1-bc4e-c420cdce16bf service nova] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:33.946611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8eb9d-ef6d-419f-9852-c9c04b360c4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:33.947427 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8eb9d-ef6d-419f-9852-c9c04b360c4a service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:33.947711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8eb9d-ef6d-419f-9852-c9c04b360c4a service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:33.947987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8eb9d-ef6d-419f-9852-c9c04b360c4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:33.965240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8eb9d-ef6d-419f-9852-c9c04b360c4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:33.967683 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8eb9d-ef6d-419f-9852-c9c04b360c4a service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:33.973611 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b6357fe7-fa0e-486c-b24a-8b4da6889501 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc864821e72d4825943e0d28ee3e1b1c', 'request_id': 'req-b6357fe7-fa0e-486c-b24a-8b4da6889501'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:33.982468 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b6357fe7-fa0e-486c-b24a-8b4da6889501 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc864821e72d4825943e0d28ee3e1b1c', 'request_id': 'req-b6357fe7-fa0e-486c-b24a-8b4da6889501'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:33.983583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 908/1840] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:33.991718 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b6357fe7-fa0e-486c-b24a-8b4da6889501 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc864821e72d4825943e0d28ee3e1b1c', 'request_id': 'req-b6357fe7-fa0e-486c-b24a-8b4da6889501'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:34.014804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b6357fe7-fa0e-486c-b24a-8b4da6889501 admin admin] The token cache is being invalidate because project fc864821e72d4825943e0d28ee3e1b1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:34.015385 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b6357fe7-fa0e-486c-b24a-8b4da6889501 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:34.016518 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b6357fe7-fa0e-486c-b24a-8b4da6889501 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:34.021088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 933/1841] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:33 2022] DELETE /identity/v3/projects/fc864821e72d4825943e0d28ee3e1b1c => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:34.025842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68fc7ce-6a68-442e-88ec-65c6735e723d None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:34.026180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68fc7ce-6a68-442e-88ec-65c6735e723d None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:34.026462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68fc7ce-6a68-442e-88ec-65c6735e723d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:34.047325 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a68fc7ce-6a68-442e-88ec-65c6735e723d None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:34.048599 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a68fc7ce-6a68-442e-88ec-65c6735e723d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:34.060179 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:34.139630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 909/1842] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:34.147740 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5b2fb-e1a8-4f94-8b0c-534a0082bfd6 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:34.149284 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5b2fb-e1a8-4f94-8b0c-534a0082bfd6 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:34.149702 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5b2fb-e1a8-4f94-8b0c-534a0082bfd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:34.175678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-0be5b2fb-e1a8-4f94-8b0c-534a0082bfd6 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:34.177861 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0be5b2fb-e1a8-4f94-8b0c-534a0082bfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:34.201656 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 934/1843] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:34.202590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68fc7ce-6a68-442e-88ec-65c6735e723d None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:34.213736 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68fc7ce-6a68-442e-88ec-65c6735e723d None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:34.229100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffa2b6-1ce0-49cf-94af-cc1633151ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:34.230103 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffa2b6-1ce0-49cf-94af-cc1633151ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:34.230403 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffa2b6-1ce0-49cf-94af-cc1633151ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:34.230665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffa2b6-1ce0-49cf-94af-cc1633151ca4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:34.231567 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ffa2b6-1ce0-49cf-94af-cc1633151ca4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:06:34.231878 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ffa2b6-1ce0-49cf-94af-cc1633151ca4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:34.233581 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ffa2b6-1ce0-49cf-94af-cc1633151ca4 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:34.269687 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 910/1844] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:06:34.275413 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5b2fb-e1a8-4f94-8b0c-534a0082bfd6 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:34.283619 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffa2b6-1ce0-49cf-94af-cc1633151ca4 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:34.285908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f9ffa2b6-1ce0-49cf-94af-cc1633151ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:34.292247 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5b2fb-e1a8-4f94-8b0c-534a0082bfd6 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:34.301225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4c0d0-b520-4e4d-8f56-46e1b027ebef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:34.302159 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4c0d0-b520-4e4d-8f56-46e1b027ebef admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:34.302444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4c0d0-b520-4e4d-8f56-46e1b027ebef admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:34.302714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4c0d0-b520-4e4d-8f56-46e1b027ebef admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:34.303467 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4c0d0-b520-4e4d-8f56-46e1b027ebef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:34.306201 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4c0d0-b520-4e4d-8f56-46e1b027ebef admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:34.312445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffa2b6-1ce0-49cf-94af-cc1633151ca4 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:34.324635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58eb7e-d2b3-4de5-8a82-bbd9af1e7546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:34.326119 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58eb7e-d2b3-4de5-8a82-bbd9af1e7546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:34.326610 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58eb7e-d2b3-4de5-8a82-bbd9af1e7546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:34.326990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58eb7e-d2b3-4de5-8a82-bbd9af1e7546 admin admin] PATH_INFO: `/v3/projects/48bcdec1c23940ec9e36fdc4b731116c` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:34.328661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58eb7e-d2b3-4de5-8a82-bbd9af1e7546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48bcdec1c23940ec9e36fdc4b731116c)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:34.331124 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58eb7e-d2b3-4de5-8a82-bbd9af1e7546 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:34.352491 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 935/1845] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:34 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:34.354218 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4c0d0-b520-4e4d-8f56-46e1b027ebef admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:34.375355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:34.406561 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5c58eb7e-d2b3-4de5-8a82-bbd9af1e7546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48bcdec1c23940ec9e36fdc4b731116c', 'request_id': 'req-5c58eb7e-d2b3-4de5-8a82-bbd9af1e7546'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:34.416549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5c58eb7e-d2b3-4de5-8a82-bbd9af1e7546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48bcdec1c23940ec9e36fdc4b731116c', 'request_id': 'req-5c58eb7e-d2b3-4de5-8a82-bbd9af1e7546'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:34.450040 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4c0d0-b520-4e4d-8f56-46e1b027ebef admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:34.540139 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8759852-d6c9-4cc9-9ccf-2171b4112951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:34.540903 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8759852-d6c9-4cc9-9ccf-2171b4112951 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:34.541185 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8759852-d6c9-4cc9-9ccf-2171b4112951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:34.541469 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8759852-d6c9-4cc9-9ccf-2171b4112951 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:34.542178 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8759852-d6c9-4cc9-9ccf-2171b4112951 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:34.544175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8759852-d6c9-4cc9-9ccf-2171b4112951 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:34.576195 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5c58eb7e-d2b3-4de5-8a82-bbd9af1e7546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48bcdec1c23940ec9e36fdc4b731116c', 'request_id': 'req-5c58eb7e-d2b3-4de5-8a82-bbd9af1e7546'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:34.581407 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 936/1846] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:34 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:34.586288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8759852-d6c9-4cc9-9ccf-2171b4112951 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:34.592861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5c58eb7e-d2b3-4de5-8a82-bbd9af1e7546 admin admin] The token cache is being invalidate because project 48bcdec1c23940ec9e36fdc4b731116c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:34.593278 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5c58eb7e-d2b3-4de5-8a82-bbd9af1e7546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:34.594011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5c58eb7e-d2b3-4de5-8a82-bbd9af1e7546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:34.598680 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 911/1847] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:34 2022] DELETE /identity/v3/projects/48bcdec1c23940ec9e36fdc4b731116c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:34.599978 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58eb7e-d2b3-4de5-8a82-bbd9af1e7546 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:34.601210 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8759852-d6c9-4cc9-9ccf-2171b4112951 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:34.601435 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5c58eb7e-d2b3-4de5-8a82-bbd9af1e7546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:34.607827 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d78eb-d9ee-4432-8896-3b99ed6fc628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:34.608725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d78eb-d9ee-4432-8896-3b99ed6fc628 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:34.609079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d78eb-d9ee-4432-8896-3b99ed6fc628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:34.609463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d78eb-d9ee-4432-8896-3b99ed6fc628 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:34.610205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d78eb-d9ee-4432-8896-3b99ed6fc628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:34.611849 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d78eb-d9ee-4432-8896-3b99ed6fc628 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:34.623907 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 937/1848] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:06:34.627338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58eb7e-d2b3-4de5-8a82-bbd9af1e7546 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:34.629222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d78eb-d9ee-4432-8896-3b99ed6fc628 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:34.632599 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1cbe40-40df-4d68-96b9-a3e6820b9f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:34.633767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1cbe40-40df-4d68-96b9-a3e6820b9f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:34.634048 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1cbe40-40df-4d68-96b9-a3e6820b9f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:34.634318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1cbe40-40df-4d68-96b9-a3e6820b9f36 admin admin] PATH_INFO: `/v3/users/dd1a9b7b4e1249b793cc8e79527ebf20` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:34.636009 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-029d78eb-d9ee-4432-8896-3b99ed6fc628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:34.636154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1cbe40-40df-4d68-96b9-a3e6820b9f36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd1a9b7b4e1249b793cc8e79527ebf20)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:34.638165 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1cbe40-40df-4d68-96b9-a3e6820b9f36 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:34.652735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d78eb-d9ee-4432-8896-3b99ed6fc628 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:34.657935 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5593f8-b13b-4b11-be24-91ea3abc1a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:34.658789 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5593f8-b13b-4b11-be24-91ea3abc1a20 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:34.659081 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5593f8-b13b-4b11-be24-91ea3abc1a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:34.659598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5593f8-b13b-4b11-be24-91ea3abc1a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:34.660340 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5593f8-b13b-4b11-be24-91ea3abc1a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:34.663141 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5593f8-b13b-4b11-be24-91ea3abc1a20 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:34.681621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 938/1849] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:34.686506 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5593f8-b13b-4b11-be24-91ea3abc1a20 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:34.713010 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5593f8-b13b-4b11-be24-91ea3abc1a20 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:34.734227 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774fb7d-c866-4c4f-8f3c-e66f1fe1b1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:34.735184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774fb7d-c866-4c4f-8f3c-e66f1fe1b1b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:34.735462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774fb7d-c866-4c4f-8f3c-e66f1fe1b1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:34.735734 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774fb7d-c866-4c4f-8f3c-e66f1fe1b1b5 admin admin] PATH_INFO: `/v3/projects/7cbd04f36a5b4b61970880205e05c431/users/37a80033688649d198f4e2dc90d70b5a/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:34.754999 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3e1cbe40-40df-4d68-96b9-a3e6820b9f36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd1a9b7b4e1249b793cc8e79527ebf20'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:34.757721 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3e1cbe40-40df-4d68-96b9-a3e6820b9f36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd1a9b7b4e1249b793cc8e79527ebf20'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:34.781762 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774fb7d-c866-4c4f-8f3c-e66f1fe1b1b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cbd04f36a5b4b61970880205e05c431, user_id=37a80033688649d198f4e2dc90d70b5a, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:34.786934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3e1cbe40-40df-4d68-96b9-a3e6820b9f36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd1a9b7b4e1249b793cc8e79527ebf20'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:34.803939 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774fb7d-c866-4c4f-8f3c-e66f1fe1b1b5 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:34.808190 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3e1cbe40-40df-4d68-96b9-a3e6820b9f36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd1a9b7b4e1249b793cc8e79527ebf20'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:34.809680 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3e1cbe40-40df-4d68-96b9-a3e6820b9f36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd1a9b7b4e1249b793cc8e79527ebf20'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:34.828283 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:34.838299 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 912/1850] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:34 2022] DELETE /identity/v3/users/dd1a9b7b4e1249b793cc8e79527ebf20 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:34.856744 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 939/1851] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:34 2022] PUT /identity/v3/projects/7cbd04f36a5b4b61970880205e05c431/users/37a80033688649d198f4e2dc90d70b5a/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:34.862080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd89a24-c23a-4e40-8fb5-f83481e41749 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:34.862373 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd89a24-c23a-4e40-8fb5-f83481e41749 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:34.862681 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd89a24-c23a-4e40-8fb5-f83481e41749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:34.908339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-1bd89a24-c23a-4e40-8fb5-f83481e41749 None None] MFA Rules not processed for user `37a80033688649d198f4e2dc90d70b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:34.912016 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1bd89a24-c23a-4e40-8fb5-f83481e41749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:35.042873 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 913/1852] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:35.047797 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774fb7d-c866-4c4f-8f3c-e66f1fe1b1b5 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:35.058125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7774fb7d-c866-4c4f-8f3c-e66f1fe1b1b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:35.221062 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774fb7d-c866-4c4f-8f3c-e66f1fe1b1b5 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:35.243870 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6995b9d-30fe-4a70-9e61-2849c8f960c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:35.244626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6995b9d-30fe-4a70-9e61-2849c8f960c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:35.245070 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6995b9d-30fe-4a70-9e61-2849c8f960c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:35.245434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6995b9d-30fe-4a70-9e61-2849c8f960c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:35.246270 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6995b9d-30fe-4a70-9e61-2849c8f960c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:35.254370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6995b9d-30fe-4a70-9e61-2849c8f960c4 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:35.261251 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd89a24-c23a-4e40-8fb5-f83481e41749 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:35.262699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1bd89a24-c23a-4e40-8fb5-f83481e41749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:35.281402 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:35.296218 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd89a24-c23a-4e40-8fb5-f83481e41749 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:35.304314 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7742b2-3e9f-48e3-a986-90f132baa1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:35.306792 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7742b2-3e9f-48e3-a986-90f132baa1a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:35.307446 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7742b2-3e9f-48e3-a986-90f132baa1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:35.307780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7742b2-3e9f-48e3-a986-90f132baa1a1 admin admin] PATH_INFO: `/v3/projects/19ee5635cbee4829a7c96140bbda9437` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:35.309564 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7742b2-3e9f-48e3-a986-90f132baa1a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19ee5635cbee4829a7c96140bbda9437)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:35.311621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7742b2-3e9f-48e3-a986-90f132baa1a1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:35.320145 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 940/1853] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:35 2022] POST /identity/v3/projects => generated 387 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:35.323936 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6995b9d-30fe-4a70-9e61-2849c8f960c4 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:35.384626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3f7742b2-3e9f-48e3-a986-90f132baa1a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19ee5635cbee4829a7c96140bbda9437', 'request_id': 'req-3f7742b2-3e9f-48e3-a986-90f132baa1a1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:35.394868 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3f7742b2-3e9f-48e3-a986-90f132baa1a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19ee5635cbee4829a7c96140bbda9437', 'request_id': 'req-3f7742b2-3e9f-48e3-a986-90f132baa1a1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:35.404320 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3f7742b2-3e9f-48e3-a986-90f132baa1a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19ee5635cbee4829a7c96140bbda9437', 'request_id': 'req-3f7742b2-3e9f-48e3-a986-90f132baa1a1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:35.427789 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6995b9d-30fe-4a70-9e61-2849c8f960c4 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:35.431193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3f7742b2-3e9f-48e3-a986-90f132baa1a1 admin admin] The token cache is being invalidate because project 19ee5635cbee4829a7c96140bbda9437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:35.431408 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3f7742b2-3e9f-48e3-a986-90f132baa1a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:35.432970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3f7742b2-3e9f-48e3-a986-90f132baa1a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:35.434571 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 914/1854] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:35 2022] DELETE /identity/v3/projects/19ee5635cbee4829a7c96140bbda9437 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:35.439949 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d14ac3-3819-4c1b-b105-c55949471144 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:35.440235 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d14ac3-3819-4c1b-b105-c55949471144 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:35.440540 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d14ac3-3819-4c1b-b105-c55949471144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:35.473277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f2d14ac3-3819-4c1b-b105-c55949471144 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:35.474734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f2d14ac3-3819-4c1b-b105-c55949471144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:35.500983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac944b4f-ebec-4b6e-81ae-a7a0dce46368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:35.502099 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac944b4f-ebec-4b6e-81ae-a7a0dce46368 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:35.502377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac944b4f-ebec-4b6e-81ae-a7a0dce46368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:35.502654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac944b4f-ebec-4b6e-81ae-a7a0dce46368 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:35.503335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac944b4f-ebec-4b6e-81ae-a7a0dce46368 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:35.505338 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac944b4f-ebec-4b6e-81ae-a7a0dce46368 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:35.542432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 941/1855] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:35 2022] POST /identity/v3/users => generated 300 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:06:35.543667 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac944b4f-ebec-4b6e-81ae-a7a0dce46368 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:35.546069 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ac944b4f-ebec-4b6e-81ae-a7a0dce46368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:35.558584 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac944b4f-ebec-4b6e-81ae-a7a0dce46368 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:35.562843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 915/1856] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:06:35.568875 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6be9b8-0017-4006-9df1-ee9f81471351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:35.569592 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6be9b8-0017-4006-9df1-ee9f81471351 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:35.569953 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6be9b8-0017-4006-9df1-ee9f81471351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:35.570231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6be9b8-0017-4006-9df1-ee9f81471351 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:35.571016 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6be9b8-0017-4006-9df1-ee9f81471351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:35.571538 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7303efa-5836-4f57-874b-cf6ee881204c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:35.571828 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7303efa-5836-4f57-874b-cf6ee881204c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:35.572106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7303efa-5836-4f57-874b-cf6ee881204c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:35.572667 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6be9b8-0017-4006-9df1-ee9f81471351 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:35.585155 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 942/1857] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:35.590718 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6be9b8-0017-4006-9df1-ee9f81471351 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:35.594303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f7303efa-5836-4f57-874b-cf6ee881204c None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:35.595496 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f7303efa-5836-4f57-874b-cf6ee881204c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:35.602467 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6be9b8-0017-4006-9df1-ee9f81471351 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:35.604086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 916/1858] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:06:35.607222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fe9e8-b779-46a7-96f0-d9f8bb5933b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:35.608180 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fe9e8-b779-46a7-96f0-d9f8bb5933b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:35.608295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7303efa-5836-4f57-874b-cf6ee881204c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:35.608484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fe9e8-b779-46a7-96f0-d9f8bb5933b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:35.608763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fe9e8-b779-46a7-96f0-d9f8bb5933b9 admin admin] PATH_INFO: `/v3/projects/e06ca3694fe94dc7ac391e7f1fa9f4b1/users/6d74aa858ccc4dcab142041e3f0a434a/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:35.612626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:35.619420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7303efa-5836-4f57-874b-cf6ee881204c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:35.625064 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24fe9e8-b779-46a7-96f0-d9f8bb5933b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e06ca3694fe94dc7ac391e7f1fa9f4b1, user_id=6d74aa858ccc4dcab142041e3f0a434a, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:35.625742 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2e759-a5ea-4d33-98c9-b9678a770d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:35.626408 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2e759-a5ea-4d33-98c9-b9678a770d6b admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:35.626686 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2e759-a5ea-4d33-98c9-b9678a770d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:35.626958 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2e759-a5ea-4d33-98c9-b9678a770d6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:35.627691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2e759-a5ea-4d33-98c9-b9678a770d6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:06:35.627886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24fe9e8-b779-46a7-96f0-d9f8bb5933b9 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:35.628058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2e759-a5ea-4d33-98c9-b9678a770d6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:35.629637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2e759-a5ea-4d33-98c9-b9678a770d6b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:35.642880 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 917/1859] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 16:06:35.650319 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 943/1860] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:35 2022] PUT /identity/v3/projects/e06ca3694fe94dc7ac391e7f1fa9f4b1/users/6d74aa858ccc4dcab142041e3f0a434a/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:35.650522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2e759-a5ea-4d33-98c9-b9678a770d6b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:35.653878 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fe9e8-b779-46a7-96f0-d9f8bb5933b9 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:35.658455 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2e759-a5ea-4d33-98c9-b9678a770d6b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:35.719822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:35.726791 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ab9f9-ce68-4a7e-8017-145fe162d5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:35.732665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ab9f9-ce68-4a7e-8017-145fe162d5cf admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:35.732948 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ab9f9-ce68-4a7e-8017-145fe162d5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:35.733217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ab9f9-ce68-4a7e-8017-145fe162d5cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:35.734007 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7ab9f9-ce68-4a7e-8017-145fe162d5cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:35.734220 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fe9e8-b779-46a7-96f0-d9f8bb5933b9 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:35.736002 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7ab9f9-ce68-4a7e-8017-145fe162d5cf admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:35.739959 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc01e3-ff39-4dde-a8c6-db30291cd846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:35.740518 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc01e3-ff39-4dde-a8c6-db30291cd846 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:35.740900 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc01e3-ff39-4dde-a8c6-db30291cd846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:35.741272 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc01e3-ff39-4dde-a8c6-db30291cd846 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:35.742609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc01e3-ff39-4dde-a8c6-db30291cd846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:35.744343 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc01e3-ff39-4dde-a8c6-db30291cd846 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:35.760474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 944/1861] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:35.764437 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc01e3-ff39-4dde-a8c6-db30291cd846 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:35.778457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc01e3-ff39-4dde-a8c6-db30291cd846 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:35.783379 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3bb19-5ae7-428f-8001-18d9730f1587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:35.784377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3bb19-5ae7-428f-8001-18d9730f1587 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:35.784903 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3bb19-5ae7-428f-8001-18d9730f1587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:35.785251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3bb19-5ae7-428f-8001-18d9730f1587 admin admin] PATH_INFO: `/v3/projects/e06ca3694fe94dc7ac391e7f1fa9f4b1/users/6d74aa858ccc4dcab142041e3f0a434a/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:35.788467 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 918/1862] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:35 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:35.788606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3bb19-5ae7-428f-8001-18d9730f1587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e06ca3694fe94dc7ac391e7f1fa9f4b1, user_id=6d74aa858ccc4dcab142041e3f0a434a, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:35.790459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ab9f9-ce68-4a7e-8017-145fe162d5cf admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:35.790632 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3bb19-5ae7-428f-8001-18d9730f1587 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:35.804655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 945/1863] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:35 2022] PUT /identity/v3/projects/e06ca3694fe94dc7ac391e7f1fa9f4b1/users/6d74aa858ccc4dcab142041e3f0a434a/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:35.815822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe95193-a01c-4c5f-8fe4-d3282232b971 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:35.816097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe95193-a01c-4c5f-8fe4-d3282232b971 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:35.816387 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe95193-a01c-4c5f-8fe4-d3282232b971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:35.873407 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5fe95193-a01c-4c5f-8fe4-d3282232b971 None None] MFA Rules not processed for user `6d74aa858ccc4dcab142041e3f0a434a`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:35.874985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5fe95193-a01c-4c5f-8fe4-d3282232b971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:35.933122 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ab9f9-ce68-4a7e-8017-145fe162d5cf admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:36.020906 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6def3-35b9-4655-ae39-c1be38e989a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:36.022608 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6def3-35b9-4655-ae39-c1be38e989a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:36.024209 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6def3-35b9-4655-ae39-c1be38e989a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:36.025011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6def3-35b9-4655-ae39-c1be38e989a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:36.026335 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6def3-35b9-4655-ae39-c1be38e989a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:36.030011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6def3-35b9-4655-ae39-c1be38e989a6 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:36.083865 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 919/1864] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:35 2022] POST /identity/v3/users => generated 292 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:36.088469 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6def3-35b9-4655-ae39-c1be38e989a6 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:36.094669 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 946/1865] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:35 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:36.115813 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6def3-35b9-4655-ae39-c1be38e989a6 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:36.121966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe95193-a01c-4c5f-8fe4-d3282232b971 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:36.125184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5fe95193-a01c-4c5f-8fe4-d3282232b971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:36.126733 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55096e4e-5c2a-4415-9106-64b4bff45c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:36.127647 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55096e4e-5c2a-4415-9106-64b4bff45c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:36.128038 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55096e4e-5c2a-4415-9106-64b4bff45c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:36.128521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55096e4e-5c2a-4415-9106-64b4bff45c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:36.129360 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55096e4e-5c2a-4415-9106-64b4bff45c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:36.140924 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55096e4e-5c2a-4415-9106-64b4bff45c2a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:36.148247 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:36.158024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 920/1866] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:36.162501 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55096e4e-5c2a-4415-9106-64b4bff45c2a admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:36.194367 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55096e4e-5c2a-4415-9106-64b4bff45c2a admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:36.212795 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a71f1-79ec-42f1-8ebb-bfb046c83dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:36.214373 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a71f1-79ec-42f1-8ebb-bfb046c83dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:36.216902 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a71f1-79ec-42f1-8ebb-bfb046c83dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:36.217308 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a71f1-79ec-42f1-8ebb-bfb046c83dd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:36.221900 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a71f1-79ec-42f1-8ebb-bfb046c83dd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:36.227459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a71f1-79ec-42f1-8ebb-bfb046c83dd2 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:36.256156 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 921/1867] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:36.261684 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a71f1-79ec-42f1-8ebb-bfb046c83dd2 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:36.294150 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a71f1-79ec-42f1-8ebb-bfb046c83dd2 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:36.315390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b2cb1-45a3-45f7-b0ab-ea1b2f286cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:36.316705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b2cb1-45a3-45f7-b0ab-ea1b2f286cac admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:36.317014 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b2cb1-45a3-45f7-b0ab-ea1b2f286cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:36.317364 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b2cb1-45a3-45f7-b0ab-ea1b2f286cac admin admin] PATH_INFO: `/v3/projects/fdf06c200eba4f1fb11b7d3fbb095201/users/ca8aa964f61144a599773b41afaa7e32/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:36.324519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe95193-a01c-4c5f-8fe4-d3282232b971 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:36.334891 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd2105-3252-466e-944e-d1de5c7e6e8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:36.335778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd2105-3252-466e-944e-d1de5c7e6e8d service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:36.336061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd2105-3252-466e-944e-d1de5c7e6e8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:36.336439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd2105-3252-466e-944e-d1de5c7e6e8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:36.353843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b2cb1-45a3-45f7-b0ab-ea1b2f286cac admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf06c200eba4f1fb11b7d3fbb095201, user_id=ca8aa964f61144a599773b41afaa7e32, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:36.357353 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b2cb1-45a3-45f7-b0ab-ea1b2f286cac admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:36.364270 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd2105-3252-466e-944e-d1de5c7e6e8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:36.366250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd2105-3252-466e-944e-d1de5c7e6e8d service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:36.381601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 947/1868] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:36.382804 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 922/1869] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:36 2022] PUT /identity/v3/projects/fdf06c200eba4f1fb11b7d3fbb095201/users/ca8aa964f61144a599773b41afaa7e32/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:36.397412 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7056a-9347-4aa2-aaad-b4774304cbbe None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:36.397703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7056a-9347-4aa2-aaad-b4774304cbbe None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:36.404954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7056a-9347-4aa2-aaad-b4774304cbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:36.437269 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b2cb1-45a3-45f7-b0ab-ea1b2f286cac admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:36.476427 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-51e7056a-9347-4aa2-aaad-b4774304cbbe None None] MFA Rules not processed for user `ca8aa964f61144a599773b41afaa7e32`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:36.479037 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-51e7056a-9347-4aa2-aaad-b4774304cbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:36.558192 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b2cb1-45a3-45f7-b0ab-ea1b2f286cac admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:36.564072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f542c6-0e06-4561-a244-117c71b60624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:36.565538 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f542c6-0e06-4561-a244-117c71b60624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:36.565962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f542c6-0e06-4561-a244-117c71b60624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:36.566481 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f542c6-0e06-4561-a244-117c71b60624 admin admin] PATH_INFO: `/v3/users/37a80033688649d198f4e2dc90d70b5a` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:36.568127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f542c6-0e06-4561-a244-117c71b60624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37a80033688649d198f4e2dc90d70b5a)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:36.570572 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f542c6-0e06-4561-a244-117c71b60624 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:36.588083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:36.630847 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 948/1870] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:36.636432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7056a-9347-4aa2-aaad-b4774304cbbe None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:36.650166 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c9f542c6-0e06-4561-a244-117c71b60624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37a80033688649d198f4e2dc90d70b5a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:36.651940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c9f542c6-0e06-4561-a244-117c71b60624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37a80033688649d198f4e2dc90d70b5a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:36.664629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7056a-9347-4aa2-aaad-b4774304cbbe None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:36.667708 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c9f542c6-0e06-4561-a244-117c71b60624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37a80033688649d198f4e2dc90d70b5a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:36.679694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5f59f-75d7-4d17-9711-3f2a2875879b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:36.680527 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5f59f-75d7-4d17-9711-3f2a2875879b admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:36.680921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5f59f-75d7-4d17-9711-3f2a2875879b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:36.681454 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5f59f-75d7-4d17-9711-3f2a2875879b admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:36.682394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5f59f-75d7-4d17-9711-3f2a2875879b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:36.684453 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5f59f-75d7-4d17-9711-3f2a2875879b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:36.688364 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c9f542c6-0e06-4561-a244-117c71b60624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37a80033688649d198f4e2dc90d70b5a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:36.689798 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c9f542c6-0e06-4561-a244-117c71b60624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37a80033688649d198f4e2dc90d70b5a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:36.719789 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 923/1871] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:36 2022] DELETE /identity/v3/users/37a80033688649d198f4e2dc90d70b5a => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:36.725975 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccac37-3d10-4d7d-93b7-5d968718a5c6 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:36.726931 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccac37-3d10-4d7d-93b7-5d968718a5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:36.727407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccac37-3d10-4d7d-93b7-5d968718a5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:36.742838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 949/1872] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:36 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:36.746519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5f59f-75d7-4d17-9711-3f2a2875879b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:36.748861 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ebd5f59f-75d7-4d17-9711-3f2a2875879b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:36.784631 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a1ccac37-3d10-4d7d-93b7-5d968718a5c6 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:36.786229 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a1ccac37-3d10-4d7d-93b7-5d968718a5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:36.875179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5f59f-75d7-4d17-9711-3f2a2875879b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:36.883703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1b774-bc07-4a21-9cf1-544adfe08107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:36.884678 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:36.885002 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1b774-bc07-4a21-9cf1-544adfe08107 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:36.885452 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1b774-bc07-4a21-9cf1-544adfe08107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:36.885866 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1b774-bc07-4a21-9cf1-544adfe08107 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:36.887305 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1b774-bc07-4a21-9cf1-544adfe08107 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:36.889554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1b774-bc07-4a21-9cf1-544adfe08107 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:36.921645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 950/1873] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:36 2022] POST /identity/v3/users => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:06:36.926626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1b774-bc07-4a21-9cf1-544adfe08107 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:36.935517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 924/1874] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:36.943561 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccac37-3d10-4d7d-93b7-5d968718a5c6 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:36.946445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a1ccac37-3d10-4d7d-93b7-5d968718a5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:36.952609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1b774-bc07-4a21-9cf1-544adfe08107 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:36.957745 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b2888-126c-4e3e-a8eb-a5f4ebfb9535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:36.958560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b2888-126c-4e3e-a8eb-a5f4ebfb9535 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:36.959377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b2888-126c-4e3e-a8eb-a5f4ebfb9535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:36.959377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b2888-126c-4e3e-a8eb-a5f4ebfb9535 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:36.960139 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b2888-126c-4e3e-a8eb-a5f4ebfb9535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:36.962124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b2888-126c-4e3e-a8eb-a5f4ebfb9535 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:36.977833 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 951/1875] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:36.982815 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b2888-126c-4e3e-a8eb-a5f4ebfb9535 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:37.001590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b2888-126c-4e3e-a8eb-a5f4ebfb9535 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:37.008062 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ec008-83be-4b8a-98db-4da5b52754ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:37.009338 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ec008-83be-4b8a-98db-4da5b52754ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:37.009764 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ec008-83be-4b8a-98db-4da5b52754ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:37.010184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ec008-83be-4b8a-98db-4da5b52754ae admin admin] PATH_INFO: `/v3/projects/69d00625d1b044909e36d205d3df48dc/users/2a6d5cf4e82e42f6878b3efb218f337e/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:37.033749 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ec008-83be-4b8a-98db-4da5b52754ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d00625d1b044909e36d205d3df48dc, user_id=2a6d5cf4e82e42f6878b3efb218f337e, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:37.046073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ec008-83be-4b8a-98db-4da5b52754ae admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:37.086844 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccac37-3d10-4d7d-93b7-5d968718a5c6 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:37.090951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 952/1876] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:36 2022] PUT /identity/v3/projects/69d00625d1b044909e36d205d3df48dc/users/2a6d5cf4e82e42f6878b3efb218f337e/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:37.095758 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ec008-83be-4b8a-98db-4da5b52754ae admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:37.192252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ec008-83be-4b8a-98db-4da5b52754ae admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:37.197698 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758fb00-b5d3-43a8-b89f-0ce8ac47642d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:37.200160 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758fb00-b5d3-43a8-b89f-0ce8ac47642d admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:37.200160 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758fb00-b5d3-43a8-b89f-0ce8ac47642d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:37.200160 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758fb00-b5d3-43a8-b89f-0ce8ac47642d admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:37.200160 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758fb00-b5d3-43a8-b89f-0ce8ac47642d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:37.201636 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758fb00-b5d3-43a8-b89f-0ce8ac47642d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:37.207600 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba79b93-92c1-4b64-92ba-527d777e07ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:37.208566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba79b93-92c1-4b64-92ba-527d777e07ea service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:37.209015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba79b93-92c1-4b64-92ba-527d777e07ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:37.209383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba79b93-92c1-4b64-92ba-527d777e07ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:37.214339 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 953/1877] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:37.222162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758fb00-b5d3-43a8-b89f-0ce8ac47642d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:37.245263 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758fb00-b5d3-43a8-b89f-0ce8ac47642d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:37.251064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba79b93-92c1-4b64-92ba-527d777e07ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:37.252771 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba79b93-92c1-4b64-92ba-527d777e07ea service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:37.260283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89fcc1-5e59-455f-a786-3f1a28e3b299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:37.261204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89fcc1-5e59-455f-a786-3f1a28e3b299 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:37.261570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89fcc1-5e59-455f-a786-3f1a28e3b299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:37.264271 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89fcc1-5e59-455f-a786-3f1a28e3b299 admin admin] PATH_INFO: `/v3/projects/69d00625d1b044909e36d205d3df48dc/users/2a6d5cf4e82e42f6878b3efb218f337e/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:37.269322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89fcc1-5e59-455f-a786-3f1a28e3b299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d00625d1b044909e36d205d3df48dc, user_id=2a6d5cf4e82e42f6878b3efb218f337e, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:37.272260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89fcc1-5e59-455f-a786-3f1a28e3b299 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:37.287936 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 925/1878] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:37.306496 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 954/1879] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:37 2022] PUT /identity/v3/projects/69d00625d1b044909e36d205d3df48dc/users/2a6d5cf4e82e42f6878b3efb218f337e/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:37.318064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abee6e-158e-4ea5-a0f5-6f387004b2c3 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:37.318355 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abee6e-158e-4ea5-a0f5-6f387004b2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:37.318629 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abee6e-158e-4ea5-a0f5-6f387004b2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:37.372456 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c7abee6e-158e-4ea5-a0f5-6f387004b2c3 None None] MFA Rules not processed for user `2a6d5cf4e82e42f6878b3efb218f337e`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:37.373661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c7abee6e-158e-4ea5-a0f5-6f387004b2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:37.533722 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 926/1880] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:37.550353 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89fcc1-5e59-455f-a786-3f1a28e3b299 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:37.552780 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-af89fcc1-5e59-455f-a786-3f1a28e3b299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:37.572835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:37.651908 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89fcc1-5e59-455f-a786-3f1a28e3b299 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:37.657464 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e290c4-3d42-40b8-95a7-b168499dfd49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:37.658363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e290c4-3d42-40b8-95a7-b168499dfd49 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:37.658792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e290c4-3d42-40b8-95a7-b168499dfd49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:37.659207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e290c4-3d42-40b8-95a7-b168499dfd49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:37.675185 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e290c4-3d42-40b8-95a7-b168499dfd49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:37.676873 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e290c4-3d42-40b8-95a7-b168499dfd49 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:37.688558 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 955/1881] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:37.809698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7abee6e-158e-4ea5-a0f5-6f387004b2c3 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:37.819782 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7abee6e-158e-4ea5-a0f5-6f387004b2c3 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:37.825584 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21e474-da00-4707-ae83-526534af72c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:37.826265 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21e474-da00-4707-ae83-526534af72c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:37.826519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21e474-da00-4707-ae83-526534af72c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:37.826822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21e474-da00-4707-ae83-526534af72c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:37.829029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bd21e474-da00-4707-ae83-526534af72c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:37.843074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e290c4-3d42-40b8-95a7-b168499dfd49 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:37.844979 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b4e290c4-3d42-40b8-95a7-b168499dfd49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:37.882338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:37.926479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21e474-da00-4707-ae83-526534af72c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:37.928079 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21e474-da00-4707-ae83-526534af72c9 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:37.937837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 927/1882] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:06:37.939257 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e290c4-3d42-40b8-95a7-b168499dfd49 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:37.943977 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19effdf-1d4e-465e-ab3e-2e27b3e76e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:37.944969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19effdf-1d4e-465e-ab3e-2e27b3e76e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:37.945276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19effdf-1d4e-465e-ab3e-2e27b3e76e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:37.945602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19effdf-1d4e-465e-ab3e-2e27b3e76e08 admin admin] PATH_INFO: `/v3/projects/7cbd04f36a5b4b61970880205e05c431` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:37.946796 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19effdf-1d4e-465e-ab3e-2e27b3e76e08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cbd04f36a5b4b61970880205e05c431)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:37.948869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19effdf-1d4e-465e-ab3e-2e27b3e76e08 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:38.004699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b19effdf-1d4e-465e-ab3e-2e27b3e76e08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cbd04f36a5b4b61970880205e05c431', 'request_id': 'req-b19effdf-1d4e-465e-ab3e-2e27b3e76e08'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:38.011923 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b19effdf-1d4e-465e-ab3e-2e27b3e76e08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cbd04f36a5b4b61970880205e05c431', 'request_id': 'req-b19effdf-1d4e-465e-ab3e-2e27b3e76e08'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:38.022099 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b19effdf-1d4e-465e-ab3e-2e27b3e76e08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cbd04f36a5b4b61970880205e05c431', 'request_id': 'req-b19effdf-1d4e-465e-ab3e-2e27b3e76e08'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:38.037615 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b19effdf-1d4e-465e-ab3e-2e27b3e76e08 admin admin] The token cache is being invalidate because project 7cbd04f36a5b4b61970880205e05c431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:38.037949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b19effdf-1d4e-465e-ab3e-2e27b3e76e08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:38.038656 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b19effdf-1d4e-465e-ab3e-2e27b3e76e08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:38.040770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 956/1883] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:37 2022] DELETE /identity/v3/projects/7cbd04f36a5b4b61970880205e05c431 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:38.044955 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21e474-da00-4707-ae83-526534af72c9 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:38.058171 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21e474-da00-4707-ae83-526534af72c9 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:38.123837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c50d0c-d8de-4e6d-a27d-0b87c4e0a0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:38.124806 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c50d0c-d8de-4e6d-a27d-0b87c4e0a0d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:38.125018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c50d0c-d8de-4e6d-a27d-0b87c4e0a0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:38.125288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c50d0c-d8de-4e6d-a27d-0b87c4e0a0d5 admin admin] PATH_INFO: `/v3/users/6d74aa858ccc4dcab142041e3f0a434a` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:38.126777 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c50d0c-d8de-4e6d-a27d-0b87c4e0a0d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d74aa858ccc4dcab142041e3f0a434a)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:38.128914 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c50d0c-d8de-4e6d-a27d-0b87c4e0a0d5 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:38.182118 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-05c50d0c-d8de-4e6d-a27d-0b87c4e0a0d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d74aa858ccc4dcab142041e3f0a434a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:38.182893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-05c50d0c-d8de-4e6d-a27d-0b87c4e0a0d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d74aa858ccc4dcab142041e3f0a434a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:38.192560 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-05c50d0c-d8de-4e6d-a27d-0b87c4e0a0d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d74aa858ccc4dcab142041e3f0a434a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:38.201202 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-05c50d0c-d8de-4e6d-a27d-0b87c4e0a0d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d74aa858ccc4dcab142041e3f0a434a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:38.201944 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-05c50d0c-d8de-4e6d-a27d-0b87c4e0a0d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d74aa858ccc4dcab142041e3f0a434a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:38.225356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 928/1884] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:38 2022] DELETE /identity/v3/users/6d74aa858ccc4dcab142041e3f0a434a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:38.514898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19effdf-1d4e-465e-ab3e-2e27b3e76e08 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:38.516968 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b19effdf-1d4e-465e-ab3e-2e27b3e76e08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:38.532053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19effdf-1d4e-465e-ab3e-2e27b3e76e08 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:38.573019 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:38.593310 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10722d48-3494-402c-8606-421bfe46b697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:38.594299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10722d48-3494-402c-8606-421bfe46b697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:38.594627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10722d48-3494-402c-8606-421bfe46b697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:38.594903 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10722d48-3494-402c-8606-421bfe46b697 admin admin] PATH_INFO: `/v3/projects/e06ca3694fe94dc7ac391e7f1fa9f4b1` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:38.596107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10722d48-3494-402c-8606-421bfe46b697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e06ca3694fe94dc7ac391e7f1fa9f4b1)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:38.598086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10722d48-3494-402c-8606-421bfe46b697 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:38.650338 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-10722d48-3494-402c-8606-421bfe46b697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e06ca3694fe94dc7ac391e7f1fa9f4b1', 'request_id': 'req-10722d48-3494-402c-8606-421bfe46b697'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:38.658368 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-10722d48-3494-402c-8606-421bfe46b697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e06ca3694fe94dc7ac391e7f1fa9f4b1', 'request_id': 'req-10722d48-3494-402c-8606-421bfe46b697'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:38.665895 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-10722d48-3494-402c-8606-421bfe46b697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e06ca3694fe94dc7ac391e7f1fa9f4b1', 'request_id': 'req-10722d48-3494-402c-8606-421bfe46b697'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:38.684792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-10722d48-3494-402c-8606-421bfe46b697 admin admin] The token cache is being invalidate because project e06ca3694fe94dc7ac391e7f1fa9f4b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:38.685100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-10722d48-3494-402c-8606-421bfe46b697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:38.685743 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-10722d48-3494-402c-8606-421bfe46b697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:38.687538 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 957/1885] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:38 2022] DELETE /identity/v3/projects/e06ca3694fe94dc7ac391e7f1fa9f4b1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:38.692369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064b7a5-033d-42db-99cf-4bf9b864f589 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:38.692754 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064b7a5-033d-42db-99cf-4bf9b864f589 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:38.693187 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064b7a5-033d-42db-99cf-4bf9b864f589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:38.713663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a064b7a5-033d-42db-99cf-4bf9b864f589 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:38.714887 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a064b7a5-033d-42db-99cf-4bf9b864f589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:38.757453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:38.790144 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 929/1886] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:06:38.798689 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d868915-55ce-4b71-af96-a88ca101ab50 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:38.798989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d868915-55ce-4b71-af96-a88ca101ab50 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:38.799263 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d868915-55ce-4b71-af96-a88ca101ab50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:38.817606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-2d868915-55ce-4b71-af96-a88ca101ab50 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:38.818819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2d868915-55ce-4b71-af96-a88ca101ab50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:38.826967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 958/1887] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:38.831345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064b7a5-033d-42db-99cf-4bf9b864f589 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:38.841987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064b7a5-033d-42db-99cf-4bf9b864f589 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:38.846661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74219631-9988-4ffb-a6ae-17fe7d4e5948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:38.847324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74219631-9988-4ffb-a6ae-17fe7d4e5948 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:38.847607 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74219631-9988-4ffb-a6ae-17fe7d4e5948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:38.847876 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74219631-9988-4ffb-a6ae-17fe7d4e5948 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:38.848656 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74219631-9988-4ffb-a6ae-17fe7d4e5948 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:06:38.848964 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74219631-9988-4ffb-a6ae-17fe7d4e5948 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:38.850567 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74219631-9988-4ffb-a6ae-17fe7d4e5948 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:38.862759 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 930/1888] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:06:38.867014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d868915-55ce-4b71-af96-a88ca101ab50 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:38.877562 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d868915-55ce-4b71-af96-a88ca101ab50 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:38.882862 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300c052-89fe-40a4-9034-a2cc23ab32ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:38.883677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300c052-89fe-40a4-9034-a2cc23ab32ec admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:38.883974 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300c052-89fe-40a4-9034-a2cc23ab32ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:38.884313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300c052-89fe-40a4-9034-a2cc23ab32ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:38.885129 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300c052-89fe-40a4-9034-a2cc23ab32ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:38.887086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300c052-89fe-40a4-9034-a2cc23ab32ec admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:38.932228 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 959/1889] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:38 2022] POST /identity/v3/projects => generated 409 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:38.936469 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74219631-9988-4ffb-a6ae-17fe7d4e5948 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:38.948942 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74219631-9988-4ffb-a6ae-17fe7d4e5948 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:39.024197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a83c5-99b9-4380-9b8c-4ec4abaa79fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:39.025049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a83c5-99b9-4380-9b8c-4ec4abaa79fd admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:39.025304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a83c5-99b9-4380-9b8c-4ec4abaa79fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:39.025577 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a83c5-99b9-4380-9b8c-4ec4abaa79fd admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:39.026284 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a83c5-99b9-4380-9b8c-4ec4abaa79fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:39.028248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a83c5-99b9-4380-9b8c-4ec4abaa79fd admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:39.062532 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300c052-89fe-40a4-9034-a2cc23ab32ec admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:39.065689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 931/1890] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:38 2022] POST /identity/v3/users => generated 305 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:39.074103 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a83c5-99b9-4380-9b8c-4ec4abaa79fd admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:39.082239 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-654a83c5-99b9-4380-9b8c-4ec4abaa79fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:39.082819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300c052-89fe-40a4-9034-a2cc23ab32ec admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:39.095701 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f8a09-28cf-4df9-bd89-c46a79ff1a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:39.096390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f8a09-28cf-4df9-bd89-c46a79ff1a95 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:39.096689 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f8a09-28cf-4df9-bd89-c46a79ff1a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:39.096963 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f8a09-28cf-4df9-bd89-c46a79ff1a95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:39.097701 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f8a09-28cf-4df9-bd89-c46a79ff1a95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:39.099946 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f8a09-28cf-4df9-bd89-c46a79ff1a95 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:39.116498 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 960/1891] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:39.117719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f8a09-28cf-4df9-bd89-c46a79ff1a95 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:39.129124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f8a09-28cf-4df9-bd89-c46a79ff1a95 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:39.136839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef7d64-6e4d-4529-ae9c-9f341f95cf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:39.137522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef7d64-6e4d-4529-ae9c-9f341f95cf59 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:39.138096 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef7d64-6e4d-4529-ae9c-9f341f95cf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:39.138472 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef7d64-6e4d-4529-ae9c-9f341f95cf59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:39.139209 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef7d64-6e4d-4529-ae9c-9f341f95cf59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:39.142345 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef7d64-6e4d-4529-ae9c-9f341f95cf59 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:39.159565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 961/1892] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:39.160979 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef7d64-6e4d-4529-ae9c-9f341f95cf59 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:39.176145 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef7d64-6e4d-4529-ae9c-9f341f95cf59 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:39.182330 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908cc120-3ec9-4487-bf72-25ef5d136226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:39.183494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cc120-3ec9-4487-bf72-25ef5d136226 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:39.183776 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cc120-3ec9-4487-bf72-25ef5d136226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:39.184051 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cc120-3ec9-4487-bf72-25ef5d136226 admin admin] PATH_INFO: `/v3/projects/ccf5acbf1e5f4cb3adf61d3d70dbb6fc/users/268d523faaf44bd282b1224503936067/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:39.202956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a83c5-99b9-4380-9b8c-4ec4abaa79fd admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:39.204021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908cc120-3ec9-4487-bf72-25ef5d136226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf5acbf1e5f4cb3adf61d3d70dbb6fc, user_id=268d523faaf44bd282b1224503936067, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:39.208620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc0796-511c-4e75-b91e-899800b5a2e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:39.209376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc0796-511c-4e75-b91e-899800b5a2e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:39.209732 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc0796-511c-4e75-b91e-899800b5a2e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:39.209841 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908cc120-3ec9-4487-bf72-25ef5d136226 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:39.210137 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc0796-511c-4e75-b91e-899800b5a2e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:39.212356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9acc0796-511c-4e75-b91e-899800b5a2e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:39.236491 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 962/1893] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:39 2022] PUT /identity/v3/projects/ccf5acbf1e5f4cb3adf61d3d70dbb6fc/users/268d523faaf44bd282b1224503936067/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:39.244617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee3e0d-33f7-444f-8131-d51d3150b821 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:39.244979 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee3e0d-33f7-444f-8131-d51d3150b821 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:39.245361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee3e0d-33f7-444f-8131-d51d3150b821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:39.284722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e1ee3e0d-33f7-444f-8131-d51d3150b821 None None] MFA Rules not processed for user `268d523faaf44bd282b1224503936067`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:39.286215 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e1ee3e0d-33f7-444f-8131-d51d3150b821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:39.303676 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc0796-511c-4e75-b91e-899800b5a2e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:39.305749 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc0796-511c-4e75-b91e-899800b5a2e8 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:39.322699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 932/1894] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:39.340860 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:39.371944 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 963/1895] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:39 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:06:39.377038 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc0796-511c-4e75-b91e-899800b5a2e8 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:39.395053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc0796-511c-4e75-b91e-899800b5a2e8 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:39.408372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:39.483463 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f08a4e-5f6a-49a9-8263-7f6d42831767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:39.484138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f08a4e-5f6a-49a9-8263-7f6d42831767 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:39.484446 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f08a4e-5f6a-49a9-8263-7f6d42831767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:39.484724 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f08a4e-5f6a-49a9-8263-7f6d42831767 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:39.485466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f08a4e-5f6a-49a9-8263-7f6d42831767 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:39.487411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f08a4e-5f6a-49a9-8263-7f6d42831767 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:39.519418 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 933/1896] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:39 2022] POST /identity/v3/projects => generated 405 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:39.523575 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee3e0d-33f7-444f-8131-d51d3150b821 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:39.543959 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee3e0d-33f7-444f-8131-d51d3150b821 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:39.621315 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064905f6-defc-4b2f-8f1f-ff04b0daa50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:39.622079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064905f6-defc-4b2f-8f1f-ff04b0daa50f admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:39.622962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064905f6-defc-4b2f-8f1f-ff04b0daa50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:39.623325 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064905f6-defc-4b2f-8f1f-ff04b0daa50f admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:39.624536 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064905f6-defc-4b2f-8f1f-ff04b0daa50f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:39.626735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064905f6-defc-4b2f-8f1f-ff04b0daa50f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:39.649128 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 964/1897] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:39 2022] POST /identity/v3/users => generated 309 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:39.653235 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f08a4e-5f6a-49a9-8263-7f6d42831767 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:39.664379 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f08a4e-5f6a-49a9-8263-7f6d42831767 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:39.670739 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026bfb1-06eb-4e3f-882a-df2272cc580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:39.671414 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026bfb1-06eb-4e3f-882a-df2272cc580e admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:39.671695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026bfb1-06eb-4e3f-882a-df2272cc580e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:39.671969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026bfb1-06eb-4e3f-882a-df2272cc580e admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:39.672733 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026bfb1-06eb-4e3f-882a-df2272cc580e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:39.674345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026bfb1-06eb-4e3f-882a-df2272cc580e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:39.687594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 934/1898] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:06:39.692842 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064905f6-defc-4b2f-8f1f-ff04b0daa50f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:39.714787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064905f6-defc-4b2f-8f1f-ff04b0daa50f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:39.725446 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265b934-9a9c-4bb8-b0fb-c4b5cd9120d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:39.727502 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265b934-9a9c-4bb8-b0fb-c4b5cd9120d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:39.728823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265b934-9a9c-4bb8-b0fb-c4b5cd9120d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:39.728823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265b934-9a9c-4bb8-b0fb-c4b5cd9120d0 admin admin] PATH_INFO: `/v3/projects/11915fcec0784d80801cfd0b4fc4f401/users/f2ca1211955640cb82cd6e488353f3fa/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:39.748247 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265b934-9a9c-4bb8-b0fb-c4b5cd9120d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11915fcec0784d80801cfd0b4fc4f401, user_id=f2ca1211955640cb82cd6e488353f3fa, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:39.751808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265b934-9a9c-4bb8-b0fb-c4b5cd9120d0 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:39.766801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 965/1899] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:39 2022] PUT /identity/v3/projects/11915fcec0784d80801cfd0b4fc4f401/users/f2ca1211955640cb82cd6e488353f3fa/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:06:39.771905 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026bfb1-06eb-4e3f-882a-df2272cc580e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:39.784853 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026bfb1-06eb-4e3f-882a-df2272cc580e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:39.869926 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f29b58-5865-449b-9a4e-d09566bb5ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:39.870607 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f29b58-5865-449b-9a4e-d09566bb5ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:39.870907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f29b58-5865-449b-9a4e-d09566bb5ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:39.871154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f29b58-5865-449b-9a4e-d09566bb5ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:39.872606 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f29b58-5865-449b-9a4e-d09566bb5ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:39.874583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f29b58-5865-449b-9a4e-d09566bb5ab9 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:39.888733 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 935/1900] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:06:39.893741 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265b934-9a9c-4bb8-b0fb-c4b5cd9120d0 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:39.905298 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265b934-9a9c-4bb8-b0fb-c4b5cd9120d0 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:39.910582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388ed60-b7cc-4a8f-a4cf-b9bc6dcbf89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:39.911886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388ed60-b7cc-4a8f-a4cf-b9bc6dcbf89f admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:39.912066 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388ed60-b7cc-4a8f-a4cf-b9bc6dcbf89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:39.912335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388ed60-b7cc-4a8f-a4cf-b9bc6dcbf89f admin admin] PATH_INFO: `/v3/projects/11915fcec0784d80801cfd0b4fc4f401/users/f2ca1211955640cb82cd6e488353f3fa/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:39.919256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388ed60-b7cc-4a8f-a4cf-b9bc6dcbf89f admin admin] RBAC: Authorizing `identity:create_grant(project_id=11915fcec0784d80801cfd0b4fc4f401, user_id=f2ca1211955640cb82cd6e488353f3fa, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:39.923779 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388ed60-b7cc-4a8f-a4cf-b9bc6dcbf89f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:39.938125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 966/1901] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:39 2022] PUT /identity/v3/projects/11915fcec0784d80801cfd0b4fc4f401/users/f2ca1211955640cb82cd6e488353f3fa/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:06:39.950614 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7916e2-04a4-4d8d-92cd-05b350f9b9d8 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:39.951269 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7916e2-04a4-4d8d-92cd-05b350f9b9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:39.951551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7916e2-04a4-4d8d-92cd-05b350f9b9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:39.982106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-bb7916e2-04a4-4d8d-92cd-05b350f9b9d8 None None] MFA Rules not processed for user `f2ca1211955640cb82cd6e488353f3fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:39.983198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bb7916e2-04a4-4d8d-92cd-05b350f9b9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:40.079946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 936/1902] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:39 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:40.092539 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388ed60-b7cc-4a8f-a4cf-b9bc6dcbf89f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:40.157882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388ed60-b7cc-4a8f-a4cf-b9bc6dcbf89f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:40.162231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f70e7e-5dd2-4b22-9646-a5a25ef85bdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:40.162880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f70e7e-5dd2-4b22-9646-a5a25ef85bdd service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:40.163129 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f70e7e-5dd2-4b22-9646-a5a25ef85bdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:40.163454 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f70e7e-5dd2-4b22-9646-a5a25ef85bdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:40.173754 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f70e7e-5dd2-4b22-9646-a5a25ef85bdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:40.175176 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f70e7e-5dd2-4b22-9646-a5a25ef85bdd service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:40.185809 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 967/1903] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:40 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:40.382431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7916e2-04a4-4d8d-92cd-05b350f9b9d8 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:40.384397 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bb7916e2-04a4-4d8d-92cd-05b350f9b9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:40.398197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7916e2-04a4-4d8d-92cd-05b350f9b9d8 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:40.402553 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5422a49-7c76-41c3-90b2-65b6237b9aeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:40.403195 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5422a49-7c76-41c3-90b2-65b6237b9aeb service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:40.403471 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5422a49-7c76-41c3-90b2-65b6237b9aeb service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:40.403744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5422a49-7c76-41c3-90b2-65b6237b9aeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:40.410039 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:40.472364 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5422a49-7c76-41c3-90b2-65b6237b9aeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:40.473962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5422a49-7c76-41c3-90b2-65b6237b9aeb service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:40.485049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 937/1904] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:40 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:47.022577 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f70e7e-5dd2-4b22-9646-a5a25ef85bdd service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:47.024867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-81f70e7e-5dd2-4b22-9646-a5a25ef85bdd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:47.074313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:47.096926 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f70e7e-5dd2-4b22-9646-a5a25ef85bdd service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:47.102474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89914c87-c720-4c75-afae-4bf982786a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:47.103351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89914c87-c720-4c75-afae-4bf982786a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:47.103633 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89914c87-c720-4c75-afae-4bf982786a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:47.103906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89914c87-c720-4c75-afae-4bf982786a4e admin admin] PATH_INFO: `/v3/users/ca8aa964f61144a599773b41afaa7e32` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:47.105158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89914c87-c720-4c75-afae-4bf982786a4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca8aa964f61144a599773b41afaa7e32)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:47.107088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89914c87-c720-4c75-afae-4bf982786a4e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:47.155989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-89914c87-c720-4c75-afae-4bf982786a4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca8aa964f61144a599773b41afaa7e32'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:47.162709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-89914c87-c720-4c75-afae-4bf982786a4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca8aa964f61144a599773b41afaa7e32'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:47.163404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-89914c87-c720-4c75-afae-4bf982786a4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca8aa964f61144a599773b41afaa7e32'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:47.173849 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-89914c87-c720-4c75-afae-4bf982786a4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca8aa964f61144a599773b41afaa7e32'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:47.174458 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-89914c87-c720-4c75-afae-4bf982786a4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca8aa964f61144a599773b41afaa7e32'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:47.191721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 968/1905] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:47 2022] DELETE /identity/v3/users/ca8aa964f61144a599773b41afaa7e32 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:47.197168 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a4ac8-38b4-4314-bc23-efb3364b02a0 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:47.197573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a4ac8-38b4-4314-bc23-efb3364b02a0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:47.197846 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a4ac8-38b4-4314-bc23-efb3364b02a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:47.216323 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c02a4ac8-38b4-4314-bc23-efb3364b02a0 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:47.217593 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c02a4ac8-38b4-4314-bc23-efb3364b02a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:47.284480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 938/1906] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:47.295033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89914c87-c720-4c75-afae-4bf982786a4e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:47.296756 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-89914c87-c720-4c75-afae-4bf982786a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:47.358143 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89914c87-c720-4c75-afae-4bf982786a4e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:47.361969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b5f0f-b4bf-4602-976b-4c91ef39594a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:47.362721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b5f0f-b4bf-4602-976b-4c91ef39594a service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:47.362991 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b5f0f-b4bf-4602-976b-4c91ef39594a service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:47.363268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b5f0f-b4bf-4602-976b-4c91ef39594a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:47.374351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b5f0f-b4bf-4602-976b-4c91ef39594a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:47.375890 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b5f0f-b4bf-4602-976b-4c91ef39594a service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:47.386205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 969/1907] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:47.740932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a4ac8-38b4-4314-bc23-efb3364b02a0 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:47.742812 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c02a4ac8-38b4-4314-bc23-efb3364b02a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:47.767167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a4ac8-38b4-4314-bc23-efb3364b02a0 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:47.772818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c573d2-df89-4dba-8e4b-0aad2d650b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:47.774024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c573d2-df89-4dba-8e4b-0aad2d650b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:47.774424 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c573d2-df89-4dba-8e4b-0aad2d650b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:47.774706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c573d2-df89-4dba-8e4b-0aad2d650b79 admin admin] PATH_INFO: `/v3/projects/fdf06c200eba4f1fb11b7d3fbb095201` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:47.776032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c573d2-df89-4dba-8e4b-0aad2d650b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdf06c200eba4f1fb11b7d3fbb095201)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:47.779505 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c573d2-df89-4dba-8e4b-0aad2d650b79 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:47.820739 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b5f0f-b4bf-4602-976b-4c91ef39594a service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:47.822719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d24b5f0f-b4bf-4602-976b-4c91ef39594a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:47.848448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b5f0f-b4bf-4602-976b-4c91ef39594a service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:47.855780 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39968d15-bf43-4caa-b730-e51c6d5c0437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:47.855780 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39968d15-bf43-4caa-b730-e51c6d5c0437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:47.855780 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39968d15-bf43-4caa-b730-e51c6d5c0437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:47.855780 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39968d15-bf43-4caa-b730-e51c6d5c0437 admin admin] PATH_INFO: `/v3/users/f656be1e69e84e7ea3679605349c63d8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:47.875598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39968d15-bf43-4caa-b730-e51c6d5c0437 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f656be1e69e84e7ea3679605349c63d8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:47.875598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39968d15-bf43-4caa-b730-e51c6d5c0437 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:47.896118 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a1c573d2-df89-4dba-8e4b-0aad2d650b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdf06c200eba4f1fb11b7d3fbb095201', 'request_id': 'req-a1c573d2-df89-4dba-8e4b-0aad2d650b79'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:47.906758 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a1c573d2-df89-4dba-8e4b-0aad2d650b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdf06c200eba4f1fb11b7d3fbb095201', 'request_id': 'req-a1c573d2-df89-4dba-8e4b-0aad2d650b79'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:47.920918 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a1c573d2-df89-4dba-8e4b-0aad2d650b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdf06c200eba4f1fb11b7d3fbb095201', 'request_id': 'req-a1c573d2-df89-4dba-8e4b-0aad2d650b79'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.007960 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a1c573d2-df89-4dba-8e4b-0aad2d650b79 admin admin] The token cache is being invalidate because project fdf06c200eba4f1fb11b7d3fbb095201 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:48.009161 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a1c573d2-df89-4dba-8e4b-0aad2d650b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.010019 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a1c573d2-df89-4dba-8e4b-0aad2d650b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.012571 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 939/1908] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:47 2022] DELETE /identity/v3/projects/fdf06c200eba4f1fb11b7d3fbb095201 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:48.022334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c573d2-df89-4dba-8e4b-0aad2d650b79 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:48.026474 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a1c573d2-df89-4dba-8e4b-0aad2d650b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:48.034149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-39968d15-bf43-4caa-b730-e51c6d5c0437 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f656be1e69e84e7ea3679605349c63d8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.046004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-39968d15-bf43-4caa-b730-e51c6d5c0437 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f656be1e69e84e7ea3679605349c63d8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.047005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-39968d15-bf43-4caa-b730-e51c6d5c0437 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f656be1e69e84e7ea3679605349c63d8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.049445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:48.075671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-39968d15-bf43-4caa-b730-e51c6d5c0437 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f656be1e69e84e7ea3679605349c63d8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.076691 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-39968d15-bf43-4caa-b730-e51c6d5c0437 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f656be1e69e84e7ea3679605349c63d8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.086197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:48.117798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 970/1909] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:47 2022] DELETE /identity/v3/users/f656be1e69e84e7ea3679605349c63d8 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:06:48.122193 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e6eff-e29b-47db-a6eb-384d42494e2d None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:48.122888 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e6eff-e29b-47db-a6eb-384d42494e2d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:48.123260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e6eff-e29b-47db-a6eb-384d42494e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:48.138745 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c573d2-df89-4dba-8e4b-0aad2d650b79 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:48.144051 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-d90e6eff-e29b-47db-a6eb-384d42494e2d None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:48.145612 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d90e6eff-e29b-47db-a6eb-384d42494e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:48.207825 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff384115-33f3-4e9d-82fb-ad0287d19aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:48.209301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff384115-33f3-4e9d-82fb-ad0287d19aa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:48.209301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff384115-33f3-4e9d-82fb-ad0287d19aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:48.209585 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff384115-33f3-4e9d-82fb-ad0287d19aa7 admin admin] PATH_INFO: `/v3/users/2a6d5cf4e82e42f6878b3efb218f337e` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:48.211009 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff384115-33f3-4e9d-82fb-ad0287d19aa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a6d5cf4e82e42f6878b3efb218f337e)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:48.213460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff384115-33f3-4e9d-82fb-ad0287d19aa7 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:48.213676 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 971/1910] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:06:48.223444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e6eff-e29b-47db-a6eb-384d42494e2d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:48.224831 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d90e6eff-e29b-47db-a6eb-384d42494e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:48.272028 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ff384115-33f3-4e9d-82fb-ad0287d19aa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a6d5cf4e82e42f6878b3efb218f337e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.272817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ff384115-33f3-4e9d-82fb-ad0287d19aa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a6d5cf4e82e42f6878b3efb218f337e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.284913 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ff384115-33f3-4e9d-82fb-ad0287d19aa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a6d5cf4e82e42f6878b3efb218f337e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.292757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e6eff-e29b-47db-a6eb-384d42494e2d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:48.295606 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ff384115-33f3-4e9d-82fb-ad0287d19aa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a6d5cf4e82e42f6878b3efb218f337e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.296295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ff384115-33f3-4e9d-82fb-ad0287d19aa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a6d5cf4e82e42f6878b3efb218f337e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.296705 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a5889-6ae3-40a2-82a1-9a103df873ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:48.297584 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a5889-6ae3-40a2-82a1-9a103df873ac service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:48.297873 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a5889-6ae3-40a2-82a1-9a103df873ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:48.298147 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a5889-6ae3-40a2-82a1-9a103df873ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:48.299949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ff7a5889-6ae3-40a2-82a1-9a103df873ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:48.318823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a5889-6ae3-40a2-82a1-9a103df873ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:48.320190 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 940/1911] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:48 2022] DELETE /identity/v3/users/2a6d5cf4e82e42f6878b3efb218f337e => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:48.320692 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a5889-6ae3-40a2-82a1-9a103df873ac service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:48.331022 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 972/1912] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:06:48.712863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff384115-33f3-4e9d-82fb-ad0287d19aa7 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:48.716179 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ff384115-33f3-4e9d-82fb-ad0287d19aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:48.737866 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a5889-6ae3-40a2-82a1-9a103df873ac service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:48.739442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ff7a5889-6ae3-40a2-82a1-9a103df873ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:48.759945 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a5889-6ae3-40a2-82a1-9a103df873ac service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:48.855236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:48.858885 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597c727-ba62-4ce5-9b13-377556f67595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:48.859774 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597c727-ba62-4ce5-9b13-377556f67595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:48.859897 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff384115-33f3-4e9d-82fb-ad0287d19aa7 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:48.860084 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597c727-ba62-4ce5-9b13-377556f67595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:48.860331 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597c727-ba62-4ce5-9b13-377556f67595 admin admin] PATH_INFO: `/v3/projects/fd544156d6b140a4914c93d4a1c6ecb7` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:48.861769 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597c727-ba62-4ce5-9b13-377556f67595 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd544156d6b140a4914c93d4a1c6ecb7)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:48.864002 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597c727-ba62-4ce5-9b13-377556f67595 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:48.868176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc228a-9f12-47c4-bfc5-d56f419f4001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:48.869654 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc228a-9f12-47c4-bfc5-d56f419f4001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:48.869948 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc228a-9f12-47c4-bfc5-d56f419f4001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:48.871137 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc228a-9f12-47c4-bfc5-d56f419f4001 admin admin] PATH_INFO: `/v3/projects/69d00625d1b044909e36d205d3df48dc` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:48.871840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc228a-9f12-47c4-bfc5-d56f419f4001 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69d00625d1b044909e36d205d3df48dc)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:48.873826 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc228a-9f12-47c4-bfc5-d56f419f4001 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:48.888148 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:48.922572 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7597c727-ba62-4ce5-9b13-377556f67595 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd544156d6b140a4914c93d4a1c6ecb7', 'request_id': 'req-7597c727-ba62-4ce5-9b13-377556f67595'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.928428 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7597c727-ba62-4ce5-9b13-377556f67595 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd544156d6b140a4914c93d4a1c6ecb7', 'request_id': 'req-7597c727-ba62-4ce5-9b13-377556f67595'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.928593 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-97bc228a-9f12-47c4-bfc5-d56f419f4001 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69d00625d1b044909e36d205d3df48dc', 'request_id': 'req-97bc228a-9f12-47c4-bfc5-d56f419f4001'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.934673 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-97bc228a-9f12-47c4-bfc5-d56f419f4001 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69d00625d1b044909e36d205d3df48dc', 'request_id': 'req-97bc228a-9f12-47c4-bfc5-d56f419f4001'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.935708 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7597c727-ba62-4ce5-9b13-377556f67595 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd544156d6b140a4914c93d4a1c6ecb7', 'request_id': 'req-7597c727-ba62-4ce5-9b13-377556f67595'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.941007 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-97bc228a-9f12-47c4-bfc5-d56f419f4001 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69d00625d1b044909e36d205d3df48dc', 'request_id': 'req-97bc228a-9f12-47c4-bfc5-d56f419f4001'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.950336 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7597c727-ba62-4ce5-9b13-377556f67595 admin admin] The token cache is being invalidate because project fd544156d6b140a4914c93d4a1c6ecb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:48.950610 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7597c727-ba62-4ce5-9b13-377556f67595 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.951347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7597c727-ba62-4ce5-9b13-377556f67595 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.953029 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 973/1913] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:48 2022] DELETE /identity/v3/projects/fd544156d6b140a4914c93d4a1c6ecb7 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:48.955253 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-97bc228a-9f12-47c4-bfc5-d56f419f4001 admin admin] The token cache is being invalidate because project 69d00625d1b044909e36d205d3df48dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:48.955534 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-97bc228a-9f12-47c4-bfc5-d56f419f4001 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.956442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-97bc228a-9f12-47c4-bfc5-d56f419f4001 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:48.956908 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597c727-ba62-4ce5-9b13-377556f67595 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:48.958115 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 941/1914] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:48 2022] DELETE /identity/v3/projects/69d00625d1b044909e36d205d3df48dc => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:48.958237 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7597c727-ba62-4ce5-9b13-377556f67595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:48.962842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e24deb-db67-4eee-a6cd-62974f0e84df None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:48.963125 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e24deb-db67-4eee-a6cd-62974f0e84df None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:48.963398 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e24deb-db67-4eee-a6cd-62974f0e84df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:48.983471 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-16e24deb-db67-4eee-a6cd-62974f0e84df None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:48.984696 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-16e24deb-db67-4eee-a6cd-62974f0e84df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:49.025048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597c727-ba62-4ce5-9b13-377556f67595 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:49.046492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 942/1915] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:49.055158 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef2aed-a989-4023-ba66-5ddad2d49db7 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:49.055573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef2aed-a989-4023-ba66-5ddad2d49db7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:49.055869 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef2aed-a989-4023-ba66-5ddad2d49db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:49.074521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c1ef2aed-a989-4023-ba66-5ddad2d49db7 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:49.075727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c1ef2aed-a989-4023-ba66-5ddad2d49db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:49.084608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708aa485-826e-4891-a77d-45cb3a0b9ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:49.085171 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 943/1916] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:06:49.085779 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708aa485-826e-4891-a77d-45cb3a0b9ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:49.086130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708aa485-826e-4891-a77d-45cb3a0b9ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:49.086411 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708aa485-826e-4891-a77d-45cb3a0b9ef1 admin admin] PATH_INFO: `/v3/users/5b7f50c74f254ebcb7913c00483ade63` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:49.087699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708aa485-826e-4891-a77d-45cb3a0b9ef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b7f50c74f254ebcb7913c00483ade63)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:49.089344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef2aed-a989-4023-ba66-5ddad2d49db7 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:49.089779 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708aa485-826e-4891-a77d-45cb3a0b9ef1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:49.098430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef2aed-a989-4023-ba66-5ddad2d49db7 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:49.104965 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071e367-34f0-4083-b392-1669c5016b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:49.105630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071e367-34f0-4083-b392-1669c5016b6d admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:49.105951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071e367-34f0-4083-b392-1669c5016b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:49.106175 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071e367-34f0-4083-b392-1669c5016b6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:49.106916 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071e367-34f0-4083-b392-1669c5016b6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:06:49.107222 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071e367-34f0-4083-b392-1669c5016b6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:49.108940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071e367-34f0-4083-b392-1669c5016b6d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:49.128896 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 944/1917] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 16:06:49.128896 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071e367-34f0-4083-b392-1669c5016b6d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:49.139576 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071e367-34f0-4083-b392-1669c5016b6d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:49.144511 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e42c38-2eab-4d2a-9f44-326b5c5807da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:49.145166 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e42c38-2eab-4d2a-9f44-326b5c5807da admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:49.145449 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e42c38-2eab-4d2a-9f44-326b5c5807da admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:49.145723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e42c38-2eab-4d2a-9f44-326b5c5807da admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:49.146418 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e42c38-2eab-4d2a-9f44-326b5c5807da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:49.148431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e42c38-2eab-4d2a-9f44-326b5c5807da admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:49.149794 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-708aa485-826e-4891-a77d-45cb3a0b9ef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b7f50c74f254ebcb7913c00483ade63'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:49.156968 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-708aa485-826e-4891-a77d-45cb3a0b9ef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b7f50c74f254ebcb7913c00483ade63'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:49.157647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-708aa485-826e-4891-a77d-45cb3a0b9ef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b7f50c74f254ebcb7913c00483ade63'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:49.159444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:49.169150 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-708aa485-826e-4891-a77d-45cb3a0b9ef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b7f50c74f254ebcb7913c00483ade63'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:49.170075 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-708aa485-826e-4891-a77d-45cb3a0b9ef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b7f50c74f254ebcb7913c00483ade63'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:49.177775 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 945/1918] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:49 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:06:49.182157 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e42c38-2eab-4d2a-9f44-326b5c5807da admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:49.183787 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-26e42c38-2eab-4d2a-9f44-326b5c5807da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:49.190816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 974/1919] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:48 2022] DELETE /identity/v3/users/5b7f50c74f254ebcb7913c00483ade63 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:06:49.235875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:49.255280 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e42c38-2eab-4d2a-9f44-326b5c5807da admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:49.320988 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c6ee1-94f1-40d6-9ec8-85a1825acb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:49.321681 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c6ee1-94f1-40d6-9ec8-85a1825acb83 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:49.322281 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c6ee1-94f1-40d6-9ec8-85a1825acb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:49.322562 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c6ee1-94f1-40d6-9ec8-85a1825acb83 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:49.323282 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c6ee1-94f1-40d6-9ec8-85a1825acb83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:49.325241 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c6ee1-94f1-40d6-9ec8-85a1825acb83 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:49.352339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 946/1920] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:49 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:06:49.356397 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708aa485-826e-4891-a77d-45cb3a0b9ef1 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:49.367021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708aa485-826e-4891-a77d-45cb3a0b9ef1 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:49.372924 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814efc9-4bac-40fa-afde-8f80a2d77c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:49.373671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814efc9-4bac-40fa-afde-8f80a2d77c2d admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:49.374013 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814efc9-4bac-40fa-afde-8f80a2d77c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:49.374352 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814efc9-4bac-40fa-afde-8f80a2d77c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:49.375150 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814efc9-4bac-40fa-afde-8f80a2d77c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:49.376898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814efc9-4bac-40fa-afde-8f80a2d77c2d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:49.388115 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 975/1921] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:49.392508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c6ee1-94f1-40d6-9ec8-85a1825acb83 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:49.403549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c6ee1-94f1-40d6-9ec8-85a1825acb83 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:49.409201 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578345d-e5c4-4542-8246-384bc788bdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:49.409878 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578345d-e5c4-4542-8246-384bc788bdf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:49.410162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578345d-e5c4-4542-8246-384bc788bdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:49.410433 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578345d-e5c4-4542-8246-384bc788bdf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:49.411193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578345d-e5c4-4542-8246-384bc788bdf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:49.412844 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578345d-e5c4-4542-8246-384bc788bdf4 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:49.427129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 947/1922] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:49.428655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814efc9-4bac-40fa-afde-8f80a2d77c2d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:49.439195 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814efc9-4bac-40fa-afde-8f80a2d77c2d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:49.443794 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e74627-05b0-4028-86da-830d7177c3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:49.444785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e74627-05b0-4028-86da-830d7177c3d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:49.445104 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e74627-05b0-4028-86da-830d7177c3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:49.445492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e74627-05b0-4028-86da-830d7177c3d3 admin admin] PATH_INFO: `/v3/projects/4de61fda8d504511ae316afd9133ae37/users/55cbc34b1ff34eb9af038065b2724507/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:49.461835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e74627-05b0-4028-86da-830d7177c3d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de61fda8d504511ae316afd9133ae37, user_id=55cbc34b1ff34eb9af038065b2724507, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:49.464589 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e74627-05b0-4028-86da-830d7177c3d3 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:49.477572 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578345d-e5c4-4542-8246-384bc788bdf4 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:49.493601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 976/1923] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:49 2022] PUT /identity/v3/projects/4de61fda8d504511ae316afd9133ae37/users/55cbc34b1ff34eb9af038065b2724507/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:49.493814 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4578345d-e5c4-4542-8246-384bc788bdf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:49.496691 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3eafc8-4d77-4693-ac22-e0e1996468e9 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:49.497333 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3eafc8-4d77-4693-ac22-e0e1996468e9 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:49.498413 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3eafc8-4d77-4693-ac22-e0e1996468e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:49.513659 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578345d-e5c4-4542-8246-384bc788bdf4 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:49.546332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-0b3eafc8-4d77-4693-ac22-e0e1996468e9 None None] MFA Rules not processed for user `55cbc34b1ff34eb9af038065b2724507`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:49.548239 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0b3eafc8-4d77-4693-ac22-e0e1996468e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:49.592293 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118e922-58fe-4474-bcd0-4176605e8387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:49.593257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118e922-58fe-4474-bcd0-4176605e8387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:49.593543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118e922-58fe-4474-bcd0-4176605e8387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:49.593823 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118e922-58fe-4474-bcd0-4176605e8387 admin admin] PATH_INFO: `/v3/projects/6752efaab1ca49cd94e6cc2e2deb5256` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:49.595074 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118e922-58fe-4474-bcd0-4176605e8387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6752efaab1ca49cd94e6cc2e2deb5256)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:49.597688 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118e922-58fe-4474-bcd0-4176605e8387 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:49.648634 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 977/1924] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:06:49.658856 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3eafc8-4d77-4693-ac22-e0e1996468e9 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:49.660447 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0b3eafc8-4d77-4693-ac22-e0e1996468e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:49.720678 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-4118e922-58fe-4474-bcd0-4176605e8387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6752efaab1ca49cd94e6cc2e2deb5256', 'request_id': 'req-4118e922-58fe-4474-bcd0-4176605e8387'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:49.738199 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-4118e922-58fe-4474-bcd0-4176605e8387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6752efaab1ca49cd94e6cc2e2deb5256', 'request_id': 'req-4118e922-58fe-4474-bcd0-4176605e8387'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:49.747309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-4118e922-58fe-4474-bcd0-4176605e8387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6752efaab1ca49cd94e6cc2e2deb5256', 'request_id': 'req-4118e922-58fe-4474-bcd0-4176605e8387'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:49.770757 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-4118e922-58fe-4474-bcd0-4176605e8387 admin admin] The token cache is being invalidate because project 6752efaab1ca49cd94e6cc2e2deb5256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:49.773081 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-4118e922-58fe-4474-bcd0-4176605e8387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:49.774077 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-4118e922-58fe-4474-bcd0-4176605e8387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:49.781466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 948/1925] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:49 2022] DELETE /identity/v3/projects/6752efaab1ca49cd94e6cc2e2deb5256 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:06:49.783937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7050db-03e6-4595-a0a4-e229a670f7ea None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:49.784504 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7050db-03e6-4595-a0a4-e229a670f7ea None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:49.785068 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7050db-03e6-4595-a0a4-e229a670f7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:49.805179 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:49.841044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5a7050db-03e6-4595-a0a4-e229a670f7ea None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:49.846950 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5a7050db-03e6-4595-a0a4-e229a670f7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:49.859823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3eafc8-4d77-4693-ac22-e0e1996468e9 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:49.977119 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:49.978365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 949/1926] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:49.980913 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63376fe-2414-4f53-a4a5-35044edab090 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:49.982646 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63376fe-2414-4f53-a4a5-35044edab090 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:49.983083 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63376fe-2414-4f53-a4a5-35044edab090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:49.985261 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cf6c0-40a8-4c89-9a87-1df83c363f78 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cba086728824ec68fb0ee90fb3f99dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:49.986989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cf6c0-40a8-4c89-9a87-1df83c363f78 service glance] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:49.987475 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cf6c0-40a8-4c89-9a87-1df83c363f78 service glance] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:49.988362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cf6c0-40a8-4c89-9a87-1df83c363f78 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:49.996169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-358cf6c0-40a8-4c89-9a87-1df83c363f78 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:50.012612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b63376fe-2414-4f53-a4a5-35044edab090 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:50.014313 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b63376fe-2414-4f53-a4a5-35044edab090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:50.032665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 950/1927] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:50.034425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63376fe-2414-4f53-a4a5-35044edab090 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:50.046924 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63376fe-2414-4f53-a4a5-35044edab090 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:50.055050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ea8df-9c2a-4ff9-bd9f-ee0e8ca23022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:50.055833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ea8df-9c2a-4ff9-bd9f-ee0e8ca23022 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:50.056160 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ea8df-9c2a-4ff9-bd9f-ee0e8ca23022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:50.056465 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ea8df-9c2a-4ff9-bd9f-ee0e8ca23022 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:50.058078 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ea8df-9c2a-4ff9-bd9f-ee0e8ca23022 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:06:50.058621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ea8df-9c2a-4ff9-bd9f-ee0e8ca23022 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:50.060342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ea8df-9c2a-4ff9-bd9f-ee0e8ca23022 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:50.074885 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358cf6c0-40a8-4c89-9a87-1df83c363f78 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:50.076469 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358cf6c0-40a8-4c89-9a87-1df83c363f78 service glance] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:50.085428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 951/1928] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:06:50.086538 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ea8df-9c2a-4ff9-bd9f-ee0e8ca23022 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:50.089466 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 978/1929] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:06:50.099417 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ea8df-9c2a-4ff9-bd9f-ee0e8ca23022 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:50.106115 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6d859-3c7b-496c-b86d-b3035800aa50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:50.106955 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6d859-3c7b-496c-b86d-b3035800aa50 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:50.107459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6d859-3c7b-496c-b86d-b3035800aa50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:50.107459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6d859-3c7b-496c-b86d-b3035800aa50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:50.108345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6d859-3c7b-496c-b86d-b3035800aa50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:50.111786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6d859-3c7b-496c-b86d-b3035800aa50 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:50.173177 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 952/1930] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:50 2022] POST /identity/v3/projects => generated 357 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:50.173673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cf6c0-40a8-4c89-9a87-1df83c363f78 service glance] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:50.192020 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cf6c0-40a8-4c89-9a87-1df83c363f78 service glance] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:50.301300 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326ded3-9d57-4056-b426-e973fb8db6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:50.302564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326ded3-9d57-4056-b426-e973fb8db6ca admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:50.302914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326ded3-9d57-4056-b426-e973fb8db6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:50.303260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326ded3-9d57-4056-b426-e973fb8db6ca admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:50.304177 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3326ded3-9d57-4056-b426-e973fb8db6ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:50.306193 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3326ded3-9d57-4056-b426-e973fb8db6ca admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:50.337767 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 979/1931] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:50 2022] POST /identity/v3/users => generated 279 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:06:50.341561 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6d859-3c7b-496c-b86d-b3035800aa50 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:50.352915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6d859-3c7b-496c-b86d-b3035800aa50 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:50.358339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f092d9-0919-40af-b6bc-a80473bc08e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:50.359015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f092d9-0919-40af-b6bc-a80473bc08e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:50.359287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f092d9-0919-40af-b6bc-a80473bc08e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:50.359555 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f092d9-0919-40af-b6bc-a80473bc08e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:50.361088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f092d9-0919-40af-b6bc-a80473bc08e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:50.362764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f092d9-0919-40af-b6bc-a80473bc08e6 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:50.376771 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 953/1932] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:06:50.378246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326ded3-9d57-4056-b426-e973fb8db6ca admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:50.392836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326ded3-9d57-4056-b426-e973fb8db6ca admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:50.397979 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651f01d-4879-4e10-bc64-bf21ed746d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:50.398646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651f01d-4879-4e10-bc64-bf21ed746d42 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:50.398920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651f01d-4879-4e10-bc64-bf21ed746d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:50.399194 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651f01d-4879-4e10-bc64-bf21ed746d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:50.399914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651f01d-4879-4e10-bc64-bf21ed746d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:50.401530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651f01d-4879-4e10-bc64-bf21ed746d42 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:50.412966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 980/1933] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:50.417701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f092d9-0919-40af-b6bc-a80473bc08e6 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:50.428993 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f092d9-0919-40af-b6bc-a80473bc08e6 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:50.434122 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b6aaf-773e-442c-8e7e-34c79f5ad3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:50.435075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b6aaf-773e-442c-8e7e-34c79f5ad3c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:50.435354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b6aaf-773e-442c-8e7e-34c79f5ad3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:50.435624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b6aaf-773e-442c-8e7e-34c79f5ad3c8 admin admin] PATH_INFO: `/v3/projects/5cccd72e8918444c94a925e052a09fb2/users/7b5127b7d5bc4a58afa9930108832697/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:50.451096 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b6aaf-773e-442c-8e7e-34c79f5ad3c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cccd72e8918444c94a925e052a09fb2, user_id=7b5127b7d5bc4a58afa9930108832697, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:50.454030 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b6aaf-773e-442c-8e7e-34c79f5ad3c8 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:50.468276 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 954/1934] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:50 2022] PUT /identity/v3/projects/5cccd72e8918444c94a925e052a09fb2/users/7b5127b7d5bc4a58afa9930108832697/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:06:50.480439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb8907-6476-495a-ae61-6de40b6df1c4 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:50.481371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb8907-6476-495a-ae61-6de40b6df1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:50.481488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb8907-6476-495a-ae61-6de40b6df1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:50.511721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-cbcb8907-6476-495a-ae61-6de40b6df1c4 None None] MFA Rules not processed for user `7b5127b7d5bc4a58afa9930108832697`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:50.513036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-cbcb8907-6476-495a-ae61-6de40b6df1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:50.588660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 981/1935] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:50 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:50.593539 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b6aaf-773e-442c-8e7e-34c79f5ad3c8 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:50.667306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b6aaf-773e-442c-8e7e-34c79f5ad3c8 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:50.672733 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a611c-0614-49d3-bcd2-1bd3c11ad6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:50.673430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a611c-0614-49d3-bcd2-1bd3c11ad6e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:50.673708 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a611c-0614-49d3-bcd2-1bd3c11ad6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:50.673965 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a611c-0614-49d3-bcd2-1bd3c11ad6e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:50.674785 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a611c-0614-49d3-bcd2-1bd3c11ad6e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:50.676920 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a611c-0614-49d3-bcd2-1bd3c11ad6e6 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:50.708110 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 955/1936] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:50 2022] POST /identity/v3/projects => generated 357 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:06:50.709433 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb8907-6476-495a-ae61-6de40b6df1c4 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:50.720759 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb8907-6476-495a-ae61-6de40b6df1c4 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:50.724584 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:50.781112 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6f5a0-32ae-498c-b574-868cd6ba4a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:50.781793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6f5a0-32ae-498c-b574-868cd6ba4a16 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:50.782159 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6f5a0-32ae-498c-b574-868cd6ba4a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:50.782357 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6f5a0-32ae-498c-b574-868cd6ba4a16 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:50.783042 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6f5a0-32ae-498c-b574-868cd6ba4a16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:50.785005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6f5a0-32ae-498c-b574-868cd6ba4a16 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:50.805577 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 982/1937] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:50 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:50.809728 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a611c-0614-49d3-bcd2-1bd3c11ad6e6 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:50.820593 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a611c-0614-49d3-bcd2-1bd3c11ad6e6 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:50.826055 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9005022-56e8-46b5-a91e-8a7beb26534d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:50.827003 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9005022-56e8-46b5-a91e-8a7beb26534d admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:50.827437 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9005022-56e8-46b5-a91e-8a7beb26534d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:50.827731 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9005022-56e8-46b5-a91e-8a7beb26534d admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:50.828741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9005022-56e8-46b5-a91e-8a7beb26534d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:50.830536 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9005022-56e8-46b5-a91e-8a7beb26534d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:50.836538 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:50.845764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 956/1938] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:50.849592 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6f5a0-32ae-498c-b574-868cd6ba4a16 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:50.859906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6f5a0-32ae-498c-b574-868cd6ba4a16 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:50.864647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50080d6-7667-4c8d-8578-bff6f0e48c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:50.865541 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50080d6-7667-4c8d-8578-bff6f0e48c15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:50.865825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50080d6-7667-4c8d-8578-bff6f0e48c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:50.866098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50080d6-7667-4c8d-8578-bff6f0e48c15 admin admin] PATH_INFO: `/v3/projects/9e2c73ce49c941138e1c17c2f5ef07ee/users/d194d57b952441c2a55d6c8f7e9d709f/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:50.883263 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50080d6-7667-4c8d-8578-bff6f0e48c15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e2c73ce49c941138e1c17c2f5ef07ee, user_id=d194d57b952441c2a55d6c8f7e9d709f, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:50.886022 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50080d6-7667-4c8d-8578-bff6f0e48c15 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:50.899768 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 983/1939] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:50 2022] PUT /identity/v3/projects/9e2c73ce49c941138e1c17c2f5ef07ee/users/d194d57b952441c2a55d6c8f7e9d709f/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:06:50.904010 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9005022-56e8-46b5-a91e-8a7beb26534d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:50.914855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9005022-56e8-46b5-a91e-8a7beb26534d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:50.978979 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e8250-40e3-4c1f-b77d-3baf76adeec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:50.979977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e8250-40e3-4c1f-b77d-3baf76adeec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:50.979977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e8250-40e3-4c1f-b77d-3baf76adeec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:50.980212 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e8250-40e3-4c1f-b77d-3baf76adeec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:50.981050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e8250-40e3-4c1f-b77d-3baf76adeec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:50.982710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e8250-40e3-4c1f-b77d-3baf76adeec6 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:50.996049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 957/1940] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:51.000265 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50080d6-7667-4c8d-8578-bff6f0e48c15 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:51.013391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50080d6-7667-4c8d-8578-bff6f0e48c15 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:51.020052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f17d65-3a6d-48dd-8930-b602f90f2da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:51.026778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f17d65-3a6d-48dd-8930-b602f90f2da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:51.026778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f17d65-3a6d-48dd-8930-b602f90f2da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:51.026778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f17d65-3a6d-48dd-8930-b602f90f2da9 admin admin] PATH_INFO: `/v3/projects/9e2c73ce49c941138e1c17c2f5ef07ee/users/d194d57b952441c2a55d6c8f7e9d709f/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:51.026778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f17d65-3a6d-48dd-8930-b602f90f2da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e2c73ce49c941138e1c17c2f5ef07ee, user_id=d194d57b952441c2a55d6c8f7e9d709f, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:51.027272 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f17d65-3a6d-48dd-8930-b602f90f2da9 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:51.038285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 984/1941] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:50 2022] PUT /identity/v3/projects/9e2c73ce49c941138e1c17c2f5ef07ee/users/d194d57b952441c2a55d6c8f7e9d709f/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:51.048202 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc39bd-5687-4caa-ba11-f306718aeb60 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:51.048519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc39bd-5687-4caa-ba11-f306718aeb60 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:51.048793 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc39bd-5687-4caa-ba11-f306718aeb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:51.075068 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-56bc39bd-5687-4caa-ba11-f306718aeb60 None None] MFA Rules not processed for user `d194d57b952441c2a55d6c8f7e9d709f`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:51.076466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-56bc39bd-5687-4caa-ba11-f306718aeb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:51.151044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 958/1942] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:51 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:51.161993 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f17d65-3a6d-48dd-8930-b602f90f2da9 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:51.164040 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d5f17d65-3a6d-48dd-8930-b602f90f2da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:51.224630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f17d65-3a6d-48dd-8930-b602f90f2da9 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:51.228586 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e52e0-4e85-4846-87c3-bb88994a7507 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:51.229273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e52e0-4e85-4846-87c3-bb88994a7507 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:51.229562 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e52e0-4e85-4846-87c3-bb88994a7507 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:51.229840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e52e0-4e85-4846-87c3-bb88994a7507 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:51.239395 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e52e0-4e85-4846-87c3-bb88994a7507 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:51.240927 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e52e0-4e85-4846-87c3-bb88994a7507 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:51.251268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 985/1943] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:51 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:51.330003 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc39bd-5687-4caa-ba11-f306718aeb60 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:51.340387 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc39bd-5687-4caa-ba11-f306718aeb60 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:51.345418 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081d3c9-20f3-49c5-a208-65271a38dad1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:51.346077 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081d3c9-20f3-49c5-a208-65271a38dad1 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:51.346499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081d3c9-20f3-49c5-a208-65271a38dad1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:51.346776 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081d3c9-20f3-49c5-a208-65271a38dad1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:51.411218 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2081d3c9-20f3-49c5-a208-65271a38dad1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:51.413085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2081d3c9-20f3-49c5-a208-65271a38dad1 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:51.427538 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 959/1944] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:51 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:54.601691 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e52e0-4e85-4846-87c3-bb88994a7507 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:54.605121 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-531e52e0-4e85-4846-87c3-bb88994a7507 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:54.666689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f301b-7f12-4e4d-b39a-77c45424ca02 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:54.667667 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f301b-7f12-4e4d-b39a-77c45424ca02 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:54.667952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f301b-7f12-4e4d-b39a-77c45424ca02 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:54.680487 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 960/1945] 10.222.0.13 () {54 vars in 921 bytes} [Sat Jun 25 16:06:54 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 25 16:06:54.692535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55d366-cd05-48f4-b83b-61371571ca29 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:54.692816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55d366-cd05-48f4-b83b-61371571ca29 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:54.694089 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55d366-cd05-48f4-b83b-61371571ca29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:54.717812 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e52e0-4e85-4846-87c3-bb88994a7507 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:54.722136 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-7d55d366-cd05-48f4-b83b-61371571ca29 None None] MFA Rules not processed for user `3622c5ccb7534e7c8e8342b4f6643bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:54.723371 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7d55d366-cd05-48f4-b83b-61371571ca29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:54.725156 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261df46-9dd2-4dfb-993b-869dc8f3208d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:54.727127 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261df46-9dd2-4dfb-993b-869dc8f3208d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:54.727758 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261df46-9dd2-4dfb-993b-869dc8f3208d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:54.728178 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261df46-9dd2-4dfb-993b-869dc8f3208d admin admin] PATH_INFO: `/v3/users/268d523faaf44bd282b1224503936067` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:54.730153 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261df46-9dd2-4dfb-993b-869dc8f3208d admin admin] RBAC: Authorizing `identity:delete_user(user_id=268d523faaf44bd282b1224503936067)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:54.732289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261df46-9dd2-4dfb-993b-869dc8f3208d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:54.737480 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:54.745148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:54.855344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b261df46-9dd2-4dfb-993b-869dc8f3208d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '268d523faaf44bd282b1224503936067'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:54.863693 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 961/1946] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Jun 25 16:06:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:54.881165 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55d366-cd05-48f4-b83b-61371571ca29 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:54.882965 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b261df46-9dd2-4dfb-993b-869dc8f3208d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '268d523faaf44bd282b1224503936067'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:54.883291 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7d55d366-cd05-48f4-b83b-61371571ca29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:54.884946 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b261df46-9dd2-4dfb-993b-869dc8f3208d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '268d523faaf44bd282b1224503936067'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:54.910233 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b261df46-9dd2-4dfb-993b-869dc8f3208d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '268d523faaf44bd282b1224503936067'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:54.911375 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b261df46-9dd2-4dfb-993b-869dc8f3208d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '268d523faaf44bd282b1224503936067'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:54.921994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55d366-cd05-48f4-b83b-61371571ca29 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:54.943185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07b81f-d726-423c-af6b-9c49299ed8bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:54.943868 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07b81f-d726-423c-af6b-9c49299ed8bd service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:54.944324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07b81f-d726-423c-af6b-9c49299ed8bd service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:54.944482 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07b81f-d726-423c-af6b-9c49299ed8bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:54.948252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-cb07b81f-d726-423c-af6b-9c49299ed8bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:54.958380 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 986/1947] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:54 2022] DELETE /identity/v3/users/268d523faaf44bd282b1224503936067 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:54.963590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ffa26-4e1c-4dde-a03b-08d2cc678bfe None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:54.963779 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ffa26-4e1c-4dde-a03b-08d2cc678bfe None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:54.965215 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ffa26-4e1c-4dde-a03b-08d2cc678bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:54.982446 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07b81f-d726-423c-af6b-9c49299ed8bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:54.984123 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07b81f-d726-423c-af6b-9c49299ed8bd service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:54.988985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-882ffa26-4e1c-4dde-a03b-08d2cc678bfe None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:54.990406 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-882ffa26-4e1c-4dde-a03b-08d2cc678bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:55.053122 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 962/1948] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:06:55.113048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 987/1949] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:55.133628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07b81f-d726-423c-af6b-9c49299ed8bd service nova] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:55.135372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-cb07b81f-d726-423c-af6b-9c49299ed8bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:55.243616 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07b81f-d726-423c-af6b-9c49299ed8bd service nova] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:55.338812 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eacb66-b7b4-4e67-82d1-b97d208b335f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:55.339671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eacb66-b7b4-4e67-82d1-b97d208b335f service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:55.339953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eacb66-b7b4-4e67-82d1-b97d208b335f service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:55.340252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eacb66-b7b4-4e67-82d1-b97d208b335f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:55.355272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eacb66-b7b4-4e67-82d1-b97d208b335f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:55.356832 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eacb66-b7b4-4e67-82d1-b97d208b335f service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:55.368474 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 963/1950] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:55.667081 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ffa26-4e1c-4dde-a03b-08d2cc678bfe None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:55.669421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-882ffa26-4e1c-4dde-a03b-08d2cc678bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:55.683832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ffa26-4e1c-4dde-a03b-08d2cc678bfe None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:55.693071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380f71d-dcd4-414c-9515-59013040b606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:55.693960 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380f71d-dcd4-414c-9515-59013040b606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:55.694255 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380f71d-dcd4-414c-9515-59013040b606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:55.694525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380f71d-dcd4-414c-9515-59013040b606 admin admin] PATH_INFO: `/v3/projects/ccf5acbf1e5f4cb3adf61d3d70dbb6fc` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:55.695853 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380f71d-dcd4-414c-9515-59013040b606 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccf5acbf1e5f4cb3adf61d3d70dbb6fc)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:55.697889 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380f71d-dcd4-414c-9515-59013040b606 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:55.747078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3380f71d-dcd4-414c-9515-59013040b606 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccf5acbf1e5f4cb3adf61d3d70dbb6fc', 'request_id': 'req-3380f71d-dcd4-414c-9515-59013040b606'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:55.752840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3380f71d-dcd4-414c-9515-59013040b606 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccf5acbf1e5f4cb3adf61d3d70dbb6fc', 'request_id': 'req-3380f71d-dcd4-414c-9515-59013040b606'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:55.759759 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3380f71d-dcd4-414c-9515-59013040b606 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccf5acbf1e5f4cb3adf61d3d70dbb6fc', 'request_id': 'req-3380f71d-dcd4-414c-9515-59013040b606'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:55.786152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3380f71d-dcd4-414c-9515-59013040b606 admin admin] The token cache is being invalidate because project ccf5acbf1e5f4cb3adf61d3d70dbb6fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:55.786467 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3380f71d-dcd4-414c-9515-59013040b606 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:55.787086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3380f71d-dcd4-414c-9515-59013040b606 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:55.788760 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 988/1951] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:55 2022] DELETE /identity/v3/projects/ccf5acbf1e5f4cb3adf61d3d70dbb6fc => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:55.792793 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eacb66-b7b4-4e67-82d1-b97d208b335f service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:55.803337 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eacb66-b7b4-4e67-82d1-b97d208b335f service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:55.861638 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:55.870107 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099eccc-afde-40a1-8faa-ddb3a7dce2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:55.871862 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099eccc-afde-40a1-8faa-ddb3a7dce2e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:55.872147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099eccc-afde-40a1-8faa-ddb3a7dce2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:55.872451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099eccc-afde-40a1-8faa-ddb3a7dce2e1 admin admin] PATH_INFO: `/v3/users/f2ca1211955640cb82cd6e488353f3fa` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:55.873766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099eccc-afde-40a1-8faa-ddb3a7dce2e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2ca1211955640cb82cd6e488353f3fa)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:55.875727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099eccc-afde-40a1-8faa-ddb3a7dce2e1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:55.924868 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f099eccc-afde-40a1-8faa-ddb3a7dce2e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2ca1211955640cb82cd6e488353f3fa'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:55.925533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f099eccc-afde-40a1-8faa-ddb3a7dce2e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2ca1211955640cb82cd6e488353f3fa'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:55.933597 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f099eccc-afde-40a1-8faa-ddb3a7dce2e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2ca1211955640cb82cd6e488353f3fa'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:55.947726 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f099eccc-afde-40a1-8faa-ddb3a7dce2e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2ca1211955640cb82cd6e488353f3fa'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:55.948515 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f099eccc-afde-40a1-8faa-ddb3a7dce2e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2ca1211955640cb82cd6e488353f3fa'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:55.967818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 964/1952] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:55 2022] DELETE /identity/v3/users/f2ca1211955640cb82cd6e488353f3fa => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:56.288712 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380f71d-dcd4-414c-9515-59013040b606 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:56.290341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3380f71d-dcd4-414c-9515-59013040b606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:56.302560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:56.307431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380f71d-dcd4-414c-9515-59013040b606 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:56.376174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69a2f4-d584-41e0-b192-e3abb2055f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:56.377485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69a2f4-d584-41e0-b192-e3abb2055f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:56.377485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69a2f4-d584-41e0-b192-e3abb2055f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:56.377689 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69a2f4-d584-41e0-b192-e3abb2055f67 admin admin] PATH_INFO: `/v3/projects/11915fcec0784d80801cfd0b4fc4f401` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:56.378921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69a2f4-d584-41e0-b192-e3abb2055f67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11915fcec0784d80801cfd0b4fc4f401)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:56.380883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69a2f4-d584-41e0-b192-e3abb2055f67 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:56.435522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9b69a2f4-d584-41e0-b192-e3abb2055f67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11915fcec0784d80801cfd0b4fc4f401', 'request_id': 'req-9b69a2f4-d584-41e0-b192-e3abb2055f67'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:56.442098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9b69a2f4-d584-41e0-b192-e3abb2055f67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11915fcec0784d80801cfd0b4fc4f401', 'request_id': 'req-9b69a2f4-d584-41e0-b192-e3abb2055f67'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:56.448957 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9b69a2f4-d584-41e0-b192-e3abb2055f67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11915fcec0784d80801cfd0b4fc4f401', 'request_id': 'req-9b69a2f4-d584-41e0-b192-e3abb2055f67'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:56.463917 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9b69a2f4-d584-41e0-b192-e3abb2055f67 admin admin] The token cache is being invalidate because project 11915fcec0784d80801cfd0b4fc4f401 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:56.464197 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9b69a2f4-d584-41e0-b192-e3abb2055f67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:56.464851 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9b69a2f4-d584-41e0-b192-e3abb2055f67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:56.466406 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 989/1953] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:56 2022] DELETE /identity/v3/projects/11915fcec0784d80801cfd0b4fc4f401 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:56.471520 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abc8e7-9e2e-490c-b17b-c67a2da7263f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:56.471744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abc8e7-9e2e-490c-b17b-c67a2da7263f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:56.472018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abc8e7-9e2e-490c-b17b-c67a2da7263f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:56.493604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-10abc8e7-9e2e-490c-b17b-c67a2da7263f None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:56.494867 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-10abc8e7-9e2e-490c-b17b-c67a2da7263f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:56.561887 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 965/1954] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:56.569689 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefed5d-171a-4a92-984d-c5236c583fa0 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:56.569968 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefed5d-171a-4a92-984d-c5236c583fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:56.570249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefed5d-171a-4a92-984d-c5236c583fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:56.590739 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7aefed5d-171a-4a92-984d-c5236c583fa0 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:56.592353 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7aefed5d-171a-4a92-984d-c5236c583fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:56.603895 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 990/1955] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:56.605341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abc8e7-9e2e-490c-b17b-c67a2da7263f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:56.615336 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abc8e7-9e2e-490c-b17b-c67a2da7263f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:56.620519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cb0aa-d922-4c3f-8278-fd98e8ae8589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:56.621197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cb0aa-d922-4c3f-8278-fd98e8ae8589 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:56.621494 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cb0aa-d922-4c3f-8278-fd98e8ae8589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:56.621774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cb0aa-d922-4c3f-8278-fd98e8ae8589 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:56.622540 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cb0aa-d922-4c3f-8278-fd98e8ae8589 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:06:56.622846 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cb0aa-d922-4c3f-8278-fd98e8ae8589 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:56.624489 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cb0aa-d922-4c3f-8278-fd98e8ae8589 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:56.640598 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 966/1956] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:06:56.641775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefed5d-171a-4a92-984d-c5236c583fa0 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:56.651081 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefed5d-171a-4a92-984d-c5236c583fa0 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:56.656236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed86f9-634b-4c6b-a0f2-7335debe8931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:56.656946 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed86f9-634b-4c6b-a0f2-7335debe8931 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:56.657197 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed86f9-634b-4c6b-a0f2-7335debe8931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:56.657487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed86f9-634b-4c6b-a0f2-7335debe8931 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:56.658188 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed86f9-634b-4c6b-a0f2-7335debe8931 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:56.660096 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed86f9-634b-4c6b-a0f2-7335debe8931 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:56.696995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 991/1957] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:56 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:56.701303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cb0aa-d922-4c3f-8278-fd98e8ae8589 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:56.711794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cb0aa-d922-4c3f-8278-fd98e8ae8589 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:56.780258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dd539-551f-4f8a-893c-6b7425114ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:56.781340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dd539-551f-4f8a-893c-6b7425114ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:56.781712 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dd539-551f-4f8a-893c-6b7425114ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:56.782134 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dd539-551f-4f8a-893c-6b7425114ca7 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:56.783249 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445dd539-551f-4f8a-893c-6b7425114ca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:56.786348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445dd539-551f-4f8a-893c-6b7425114ca7 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:56.795980 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:56.815412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 967/1958] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:56 2022] POST /identity/v3/users => generated 291 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:56.816711 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed86f9-634b-4c6b-a0f2-7335debe8931 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:56.827255 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed86f9-634b-4c6b-a0f2-7335debe8931 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:56.831872 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e65e15-cd4b-4ec6-b153-bc7e142d18c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:56.832540 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e65e15-cd4b-4ec6-b153-bc7e142d18c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:56.832843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e65e15-cd4b-4ec6-b153-bc7e142d18c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:56.833095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e65e15-cd4b-4ec6-b153-bc7e142d18c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:56.833842 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e65e15-cd4b-4ec6-b153-bc7e142d18c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:56.835661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e65e15-cd4b-4ec6-b153-bc7e142d18c5 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:56.846194 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 992/1959] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:56.850197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dd539-551f-4f8a-893c-6b7425114ca7 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:56.866750 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dd539-551f-4f8a-893c-6b7425114ca7 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:56.872672 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949d043-a0bf-4268-8180-be56c391051b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:56.873348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949d043-a0bf-4268-8180-be56c391051b admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:56.873597 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949d043-a0bf-4268-8180-be56c391051b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:56.873878 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949d043-a0bf-4268-8180-be56c391051b admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:56.874604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949d043-a0bf-4268-8180-be56c391051b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:56.876200 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949d043-a0bf-4268-8180-be56c391051b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:56.887159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 968/1960] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:56.891861 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e65e15-cd4b-4ec6-b153-bc7e142d18c5 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:56.902183 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e65e15-cd4b-4ec6-b153-bc7e142d18c5 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:56.907128 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eefb608-2280-4106-ab70-ec79f75e59cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:56.908019 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefb608-2280-4106-ab70-ec79f75e59cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:56.908298 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefb608-2280-4106-ab70-ec79f75e59cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:56.908608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefb608-2280-4106-ab70-ec79f75e59cc admin admin] PATH_INFO: `/v3/projects/2bc43547000f49d6a18a7270a6a4a6ea/users/7210b08f05494d18b7830cf850023ba0/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:56.924625 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eefb608-2280-4106-ab70-ec79f75e59cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bc43547000f49d6a18a7270a6a4a6ea, user_id=7210b08f05494d18b7830cf850023ba0, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:56.927273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eefb608-2280-4106-ab70-ec79f75e59cc admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:56.943372 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 993/1961] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:56 2022] PUT /identity/v3/projects/2bc43547000f49d6a18a7270a6a4a6ea/users/7210b08f05494d18b7830cf850023ba0/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:56.961102 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ccd6d-8e82-43f9-9221-e0a0bafeec5b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:56.961385 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ccd6d-8e82-43f9-9221-e0a0bafeec5b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:56.961681 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ccd6d-8e82-43f9-9221-e0a0bafeec5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:56.989941 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d55ccd6d-8e82-43f9-9221-e0a0bafeec5b None None] MFA Rules not processed for user `7210b08f05494d18b7830cf850023ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:56.991171 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d55ccd6d-8e82-43f9-9221-e0a0bafeec5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:57.060472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 969/1962] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:57.065818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eefb608-2280-4106-ab70-ec79f75e59cc admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:57.115942 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:57.152810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eefb608-2280-4106-ab70-ec79f75e59cc admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:57.157503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c14c7-3d0b-4ba5-a967-c995020e2394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:57.158260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c14c7-3d0b-4ba5-a967-c995020e2394 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:57.158606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c14c7-3d0b-4ba5-a967-c995020e2394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:57.158942 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c14c7-3d0b-4ba5-a967-c995020e2394 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:57.159750 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c14c7-3d0b-4ba5-a967-c995020e2394 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:57.161681 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c14c7-3d0b-4ba5-a967-c995020e2394 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:57.189494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 994/1963] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:06:57 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:57.193529 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ccd6d-8e82-43f9-9221-e0a0bafeec5b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:57.207908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ccd6d-8e82-43f9-9221-e0a0bafeec5b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:57.273210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31eea6-9b4a-4207-965f-94ce43fc2a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:57.273746 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31eea6-9b4a-4207-965f-94ce43fc2a7e admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:57.274332 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31eea6-9b4a-4207-965f-94ce43fc2a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:57.274332 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31eea6-9b4a-4207-965f-94ce43fc2a7e admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:57.275032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe31eea6-9b4a-4207-965f-94ce43fc2a7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:57.276985 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe31eea6-9b4a-4207-965f-94ce43fc2a7e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:57.306066 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 970/1964] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:06:57 2022] POST /identity/v3/users => generated 297 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:06:57.309943 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c14c7-3d0b-4ba5-a967-c995020e2394 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:57.323290 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c14c7-3d0b-4ba5-a967-c995020e2394 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:57.329584 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfc1fb-15f4-4116-9e24-4f5e2d2c210d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:57.330276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfc1fb-15f4-4116-9e24-4f5e2d2c210d admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:57.330563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfc1fb-15f4-4116-9e24-4f5e2d2c210d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:57.330840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfc1fb-15f4-4116-9e24-4f5e2d2c210d admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:57.331601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfc1fb-15f4-4116-9e24-4f5e2d2c210d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:57.333368 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfc1fb-15f4-4116-9e24-4f5e2d2c210d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:57.346835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 995/1965] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:57.351606 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31eea6-9b4a-4207-965f-94ce43fc2a7e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:57.363184 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31eea6-9b4a-4207-965f-94ce43fc2a7e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:57.375437 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51df81-7905-4368-a6ed-5717f66e3849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:57.376386 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51df81-7905-4368-a6ed-5717f66e3849 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:57.376708 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51df81-7905-4368-a6ed-5717f66e3849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:57.376968 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51df81-7905-4368-a6ed-5717f66e3849 admin admin] PATH_INFO: `/v3/projects/cab9d12d1d1048d7b5b419bd526b30f9/users/8b26b6be22774b529a078411fadb1039/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:57.395375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51df81-7905-4368-a6ed-5717f66e3849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab9d12d1d1048d7b5b419bd526b30f9, user_id=8b26b6be22774b529a078411fadb1039, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:57.398139 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51df81-7905-4368-a6ed-5717f66e3849 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:57.414797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 971/1966] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:57 2022] PUT /identity/v3/projects/cab9d12d1d1048d7b5b419bd526b30f9/users/8b26b6be22774b529a078411fadb1039/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:57.418773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfc1fb-15f4-4116-9e24-4f5e2d2c210d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:57.429327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfc1fb-15f4-4116-9e24-4f5e2d2c210d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:57.529324 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a62c9-efba-47d3-8bc7-c903569ede19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:57.530145 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a62c9-efba-47d3-8bc7-c903569ede19 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:57.530527 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a62c9-efba-47d3-8bc7-c903569ede19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:57.530941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a62c9-efba-47d3-8bc7-c903569ede19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:57.531783 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a62c9-efba-47d3-8bc7-c903569ede19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:57.535105 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a62c9-efba-47d3-8bc7-c903569ede19 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:57.550565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 996/1967] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:06:57.555947 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51df81-7905-4368-a6ed-5717f66e3849 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:57.574593 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51df81-7905-4368-a6ed-5717f66e3849 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:57.589175 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1fe32-c2e5-4d15-a0a6-2e63ff6536fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:57.590099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1fe32-c2e5-4d15-a0a6-2e63ff6536fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:57.590378 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1fe32-c2e5-4d15-a0a6-2e63ff6536fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:57.590652 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1fe32-c2e5-4d15-a0a6-2e63ff6536fc admin admin] PATH_INFO: `/v3/projects/cab9d12d1d1048d7b5b419bd526b30f9/users/8b26b6be22774b529a078411fadb1039/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:57.601709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1fe32-c2e5-4d15-a0a6-2e63ff6536fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab9d12d1d1048d7b5b419bd526b30f9, user_id=8b26b6be22774b529a078411fadb1039, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:57.604394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1fe32-c2e5-4d15-a0a6-2e63ff6536fc admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:57.627437 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 972/1968] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:06:57 2022] PUT /identity/v3/projects/cab9d12d1d1048d7b5b419bd526b30f9/users/8b26b6be22774b529a078411fadb1039/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:57.638332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efc25c-e8b5-4cf4-986c-897568e968c7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:57.638798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efc25c-e8b5-4cf4-986c-897568e968c7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:57.639264 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efc25c-e8b5-4cf4-986c-897568e968c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:57.673655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c2efc25c-e8b5-4cf4-986c-897568e968c7 None None] MFA Rules not processed for user `8b26b6be22774b529a078411fadb1039`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:57.674233 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c2efc25c-e8b5-4cf4-986c-897568e968c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:57.776128 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 997/1969] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:57.784275 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1fe32-c2e5-4d15-a0a6-2e63ff6536fc admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:57.786097 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1af1fe32-c2e5-4d15-a0a6-2e63ff6536fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:57.814389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:57.862357 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1fe32-c2e5-4d15-a0a6-2e63ff6536fc admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:57.866711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053a402-a5a9-4310-bfd4-41d112afbeed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:57.867076 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053a402-a5a9-4310-bfd4-41d112afbeed service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:57.867351 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053a402-a5a9-4310-bfd4-41d112afbeed service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:57.867627 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053a402-a5a9-4310-bfd4-41d112afbeed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:57.877879 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053a402-a5a9-4310-bfd4-41d112afbeed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:57.879722 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053a402-a5a9-4310-bfd4-41d112afbeed service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:57.889918 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 973/1970] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:57.987122 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efc25c-e8b5-4cf4-986c-897568e968c7 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:57.999582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efc25c-e8b5-4cf4-986c-897568e968c7 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:58.015708 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6482a3d1-d6fc-4025-9bdf-b73e90d409af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:58.017737 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482a3d1-d6fc-4025-9bdf-b73e90d409af service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:58.018258 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482a3d1-d6fc-4025-9bdf-b73e90d409af service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:58.018751 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6482a3d1-d6fc-4025-9bdf-b73e90d409af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:58.047354 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:58.083407 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6482a3d1-d6fc-4025-9bdf-b73e90d409af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:58.085350 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6482a3d1-d6fc-4025-9bdf-b73e90d409af service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:58.093986 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 998/1971] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:58.385036 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053a402-a5a9-4310-bfd4-41d112afbeed service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:58.471215 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053a402-a5a9-4310-bfd4-41d112afbeed service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:58.477733 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844bd67-9670-40cb-920d-29b8c1910d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:58.478625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844bd67-9670-40cb-920d-29b8c1910d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:58.478902 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844bd67-9670-40cb-920d-29b8c1910d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:58.479173 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844bd67-9670-40cb-920d-29b8c1910d72 admin admin] PATH_INFO: `/v3/users/7210b08f05494d18b7830cf850023ba0` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:58.480392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844bd67-9670-40cb-920d-29b8c1910d72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7210b08f05494d18b7830cf850023ba0)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:58.482392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844bd67-9670-40cb-920d-29b8c1910d72 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:58.552209 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d844bd67-9670-40cb-920d-29b8c1910d72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7210b08f05494d18b7830cf850023ba0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:58.552908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d844bd67-9670-40cb-920d-29b8c1910d72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7210b08f05494d18b7830cf850023ba0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:58.562548 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d844bd67-9670-40cb-920d-29b8c1910d72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7210b08f05494d18b7830cf850023ba0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:58.571897 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d844bd67-9670-40cb-920d-29b8c1910d72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7210b08f05494d18b7830cf850023ba0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:58.572865 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d844bd67-9670-40cb-920d-29b8c1910d72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7210b08f05494d18b7830cf850023ba0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:58.601246 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 974/1972] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:58 2022] DELETE /identity/v3/users/7210b08f05494d18b7830cf850023ba0 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:06:58.606078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d829a86-3988-443c-8299-8adbb2d18d8f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:58.606382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d829a86-3988-443c-8299-8adbb2d18d8f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:58.606645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d829a86-3988-443c-8299-8adbb2d18d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:58.639371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5d829a86-3988-443c-8299-8adbb2d18d8f None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:06:58.642870 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5d829a86-3988-443c-8299-8adbb2d18d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:58.722450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 999/1973] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:06:58.732640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844bd67-9670-40cb-920d-29b8c1910d72 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:58.734242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d844bd67-9670-40cb-920d-29b8c1910d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:58.798544 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:06:58.804556 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844bd67-9670-40cb-920d-29b8c1910d72 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:58.809666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bca537-cfb6-4597-9b65-1193c3326cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:58.810330 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bca537-cfb6-4597-9b65-1193c3326cfb service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:58.810835 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bca537-cfb6-4597-9b65-1193c3326cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:58.811233 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bca537-cfb6-4597-9b65-1193c3326cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:58.821615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bca537-cfb6-4597-9b65-1193c3326cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:58.823074 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bca537-cfb6-4597-9b65-1193c3326cfb service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:58.832964 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 975/1974] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:06:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:06:59.379384 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d829a86-3988-443c-8299-8adbb2d18d8f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:59.381647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5d829a86-3988-443c-8299-8adbb2d18d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:06:59.397413 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d829a86-3988-443c-8299-8adbb2d18d8f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:59.405078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97385f2-fa3b-4d43-9d3f-c754fc657d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:59.406042 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97385f2-fa3b-4d43-9d3f-c754fc657d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:59.406414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97385f2-fa3b-4d43-9d3f-c754fc657d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:59.406884 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97385f2-fa3b-4d43-9d3f-c754fc657d1f admin admin] PATH_INFO: `/v3/projects/2bc43547000f49d6a18a7270a6a4a6ea` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:59.408392 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97385f2-fa3b-4d43-9d3f-c754fc657d1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bc43547000f49d6a18a7270a6a4a6ea)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:59.423451 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97385f2-fa3b-4d43-9d3f-c754fc657d1f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:59.494366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d97385f2-fa3b-4d43-9d3f-c754fc657d1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bc43547000f49d6a18a7270a6a4a6ea', 'request_id': 'req-d97385f2-fa3b-4d43-9d3f-c754fc657d1f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:59.500596 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d97385f2-fa3b-4d43-9d3f-c754fc657d1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bc43547000f49d6a18a7270a6a4a6ea', 'request_id': 'req-d97385f2-fa3b-4d43-9d3f-c754fc657d1f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:59.509442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d97385f2-fa3b-4d43-9d3f-c754fc657d1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bc43547000f49d6a18a7270a6a4a6ea', 'request_id': 'req-d97385f2-fa3b-4d43-9d3f-c754fc657d1f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:59.525004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d97385f2-fa3b-4d43-9d3f-c754fc657d1f admin admin] The token cache is being invalidate because project 2bc43547000f49d6a18a7270a6a4a6ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:06:59.527902 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d97385f2-fa3b-4d43-9d3f-c754fc657d1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:59.528822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d97385f2-fa3b-4d43-9d3f-c754fc657d1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:59.530565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1000/1975] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:59 2022] DELETE /identity/v3/projects/2bc43547000f49d6a18a7270a6a4a6ea => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:59.534666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bca537-cfb6-4597-9b65-1193c3326cfb service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:59.551659 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bca537-cfb6-4597-9b65-1193c3326cfb service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:06:59.608118 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40277ad4-df75-42cf-bd72-3b09d947071e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:06:59.608955 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40277ad4-df75-42cf-bd72-3b09d947071e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:06:59.609230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40277ad4-df75-42cf-bd72-3b09d947071e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:06:59.609507 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40277ad4-df75-42cf-bd72-3b09d947071e admin admin] PATH_INFO: `/v3/users/8b26b6be22774b529a078411fadb1039` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:06:59.610714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40277ad4-df75-42cf-bd72-3b09d947071e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b26b6be22774b529a078411fadb1039)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:06:59.612678 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40277ad4-df75-42cf-bd72-3b09d947071e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:06:59.666371 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-40277ad4-df75-42cf-bd72-3b09d947071e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b26b6be22774b529a078411fadb1039'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:59.667038 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-40277ad4-df75-42cf-bd72-3b09d947071e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b26b6be22774b529a078411fadb1039'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:59.677150 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-40277ad4-df75-42cf-bd72-3b09d947071e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b26b6be22774b529a078411fadb1039'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:59.684149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-40277ad4-df75-42cf-bd72-3b09d947071e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b26b6be22774b529a078411fadb1039'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:59.684791 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-40277ad4-df75-42cf-bd72-3b09d947071e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b26b6be22774b529a078411fadb1039'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:06:59.701672 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 976/1976] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:06:59 2022] DELETE /identity/v3/users/8b26b6be22774b529a078411fadb1039 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:06:59.993536 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97385f2-fa3b-4d43-9d3f-c754fc657d1f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:06:59.994978 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d97385f2-fa3b-4d43-9d3f-c754fc657d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:00.009046 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97385f2-fa3b-4d43-9d3f-c754fc657d1f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:00.064612 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:00.079665 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da137b-0c08-4676-b8aa-f205758b55b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:00.080620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da137b-0c08-4676-b8aa-f205758b55b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:00.080908 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da137b-0c08-4676-b8aa-f205758b55b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:00.081184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da137b-0c08-4676-b8aa-f205758b55b2 admin admin] PATH_INFO: `/v3/projects/cab9d12d1d1048d7b5b419bd526b30f9` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:00.082485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da137b-0c08-4676-b8aa-f205758b55b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cab9d12d1d1048d7b5b419bd526b30f9)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:00.084440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da137b-0c08-4676-b8aa-f205758b55b2 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:00.139682 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-49da137b-0c08-4676-b8aa-f205758b55b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cab9d12d1d1048d7b5b419bd526b30f9', 'request_id': 'req-49da137b-0c08-4676-b8aa-f205758b55b2'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:00.145874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-49da137b-0c08-4676-b8aa-f205758b55b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cab9d12d1d1048d7b5b419bd526b30f9', 'request_id': 'req-49da137b-0c08-4676-b8aa-f205758b55b2'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:00.152834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-49da137b-0c08-4676-b8aa-f205758b55b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cab9d12d1d1048d7b5b419bd526b30f9', 'request_id': 'req-49da137b-0c08-4676-b8aa-f205758b55b2'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:00.167294 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-49da137b-0c08-4676-b8aa-f205758b55b2 admin admin] The token cache is being invalidate because project cab9d12d1d1048d7b5b419bd526b30f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:00.167607 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-49da137b-0c08-4676-b8aa-f205758b55b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:00.168307 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-49da137b-0c08-4676-b8aa-f205758b55b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:00.169842 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1001/1977] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:06:59 2022] DELETE /identity/v3/projects/cab9d12d1d1048d7b5b419bd526b30f9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:00.176364 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ff939-ee39-4229-84d5-c47ba8249afa None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:00.176895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ff939-ee39-4229-84d5-c47ba8249afa None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:00.177098 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ff939-ee39-4229-84d5-c47ba8249afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:00.197108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b53ff939-ee39-4229-84d5-c47ba8249afa None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:00.198404 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b53ff939-ee39-4229-84d5-c47ba8249afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:00.318496 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 977/1978] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:07:00.327829 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fef6e7-5ad4-480a-8b8c-8c0460049c38 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:00.328122 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fef6e7-5ad4-480a-8b8c-8c0460049c38 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:00.328433 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fef6e7-5ad4-480a-8b8c-8c0460049c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:00.372011 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a0fef6e7-5ad4-480a-8b8c-8c0460049c38 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:00.373396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a0fef6e7-5ad4-480a-8b8c-8c0460049c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:00.390778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1002/1979] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:00.392261 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ff939-ee39-4229-84d5-c47ba8249afa None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:00.419188 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ff939-ee39-4229-84d5-c47ba8249afa None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:00.436199 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eae9a6-eaa0-4ff4-ace9-65379608e202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:00.437041 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eae9a6-eaa0-4ff4-ace9-65379608e202 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:00.437414 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eae9a6-eaa0-4ff4-ace9-65379608e202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:00.437753 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eae9a6-eaa0-4ff4-ace9-65379608e202 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:00.438616 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eae9a6-eaa0-4ff4-ace9-65379608e202 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:07:00.438985 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eae9a6-eaa0-4ff4-ace9-65379608e202 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:00.443110 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eae9a6-eaa0-4ff4-ace9-65379608e202 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:00.450672 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:00.518433 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 978/1980] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 16:07:00.520120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fef6e7-5ad4-480a-8b8c-8c0460049c38 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:00.541769 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fef6e7-5ad4-480a-8b8c-8c0460049c38 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:00.549711 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf18c75-03b9-4e56-9724-585a8eecae3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:00.550863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf18c75-03b9-4e56-9724-585a8eecae3d admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:00.550863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf18c75-03b9-4e56-9724-585a8eecae3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:00.551136 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf18c75-03b9-4e56-9724-585a8eecae3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:00.552037 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf18c75-03b9-4e56-9724-585a8eecae3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:00.556990 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf18c75-03b9-4e56-9724-585a8eecae3d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:00.625437 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1003/1981] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:00 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:00.627427 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eae9a6-eaa0-4ff4-ace9-65379608e202 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:00.642416 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eae9a6-eaa0-4ff4-ace9-65379608e202 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:00.799604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f6068-4da7-432e-8bf7-ec853abca1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:00.800045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f6068-4da7-432e-8bf7-ec853abca1be admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:00.800387 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f6068-4da7-432e-8bf7-ec853abca1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:00.800770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f6068-4da7-432e-8bf7-ec853abca1be admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:00.801625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f6068-4da7-432e-8bf7-ec853abca1be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:00.804393 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f6068-4da7-432e-8bf7-ec853abca1be admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:00.840843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 979/1982] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:00 2022] POST /identity/v3/users => generated 291 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:00.844634 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf18c75-03b9-4e56-9724-585a8eecae3d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:00.882916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf18c75-03b9-4e56-9724-585a8eecae3d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:00.907007 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976768fd-232b-4472-bf7a-5f4ef55fe19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:00.910512 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976768fd-232b-4472-bf7a-5f4ef55fe19c admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:00.910512 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976768fd-232b-4472-bf7a-5f4ef55fe19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:00.910512 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976768fd-232b-4472-bf7a-5f4ef55fe19c admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:00.916711 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976768fd-232b-4472-bf7a-5f4ef55fe19c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:00.925027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976768fd-232b-4472-bf7a-5f4ef55fe19c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:00.965706 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1004/1983] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:00.969402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f6068-4da7-432e-8bf7-ec853abca1be admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:00.996621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f6068-4da7-432e-8bf7-ec853abca1be admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:01.006573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6787c4-84f2-4271-a2c4-70b5ecb0bdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:01.007346 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6787c4-84f2-4271-a2c4-70b5ecb0bdb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:01.007765 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6787c4-84f2-4271-a2c4-70b5ecb0bdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:01.008126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6787c4-84f2-4271-a2c4-70b5ecb0bdb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:01.009292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6787c4-84f2-4271-a2c4-70b5ecb0bdb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:01.011039 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6787c4-84f2-4271-a2c4-70b5ecb0bdb2 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:01.027506 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 980/1984] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:01.029806 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976768fd-232b-4472-bf7a-5f4ef55fe19c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:01.042286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976768fd-232b-4472-bf7a-5f4ef55fe19c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:01.049388 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6bc89-cd2f-4c45-bf9a-4a23500fb65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:01.050317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6bc89-cd2f-4c45-bf9a-4a23500fb65d admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:01.050597 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6bc89-cd2f-4c45-bf9a-4a23500fb65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:01.051041 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6bc89-cd2f-4c45-bf9a-4a23500fb65d admin admin] PATH_INFO: `/v3/projects/5c26b071c98d4801bf1964bd53ea3b23/users/632f774bcff747e995703dcd27935cf0/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:01.071757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6bc89-cd2f-4c45-bf9a-4a23500fb65d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c26b071c98d4801bf1964bd53ea3b23, user_id=632f774bcff747e995703dcd27935cf0, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:01.074667 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6bc89-cd2f-4c45-bf9a-4a23500fb65d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:01.088879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1005/1985] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:01 2022] PUT /identity/v3/projects/5c26b071c98d4801bf1964bd53ea3b23/users/632f774bcff747e995703dcd27935cf0/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:01.099210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6164a74-e64a-42d6-b57f-52743e8f3a02 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:01.099409 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6164a74-e64a-42d6-b57f-52743e8f3a02 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:01.099698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6164a74-e64a-42d6-b57f-52743e8f3a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:01.128049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f6164a74-e64a-42d6-b57f-52743e8f3a02 None None] MFA Rules not processed for user `632f774bcff747e995703dcd27935cf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:01.130037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f6164a74-e64a-42d6-b57f-52743e8f3a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:01.206153 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 981/1986] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:01.216921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6bc89-cd2f-4c45-bf9a-4a23500fb65d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:01.218682 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0cf6bc89-cd2f-4c45-bf9a-4a23500fb65d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:01.278788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6bc89-cd2f-4c45-bf9a-4a23500fb65d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:01.282694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e42b60-3f0b-44e3-92d6-c49e1a6acd66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:01.283349 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e42b60-3f0b-44e3-92d6-c49e1a6acd66 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:01.283635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e42b60-3f0b-44e3-92d6-c49e1a6acd66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:01.283908 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e42b60-3f0b-44e3-92d6-c49e1a6acd66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:01.292728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e42b60-3f0b-44e3-92d6-c49e1a6acd66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:01.294154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e42b60-3f0b-44e3-92d6-c49e1a6acd66 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:01.303130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1006/1987] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:06.032798 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6164a74-e64a-42d6-b57f-52743e8f3a02 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:06.145167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6164a74-e64a-42d6-b57f-52743e8f3a02 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:06.150995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc72e65-d6f7-4973-8ee5-dc0577e2dcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:06.156472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc72e65-d6f7-4973-8ee5-dc0577e2dcf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:06.157446 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc72e65-d6f7-4973-8ee5-dc0577e2dcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:06.157812 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc72e65-d6f7-4973-8ee5-dc0577e2dcf9 admin admin] PATH_INFO: `/v3/users/632f774bcff747e995703dcd27935cf0` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:06.159567 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc72e65-d6f7-4973-8ee5-dc0577e2dcf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=632f774bcff747e995703dcd27935cf0)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:06.162183 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc72e65-d6f7-4973-8ee5-dc0577e2dcf9 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:06.170140 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:06.237268 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7dc72e65-d6f7-4973-8ee5-dc0577e2dcf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '632f774bcff747e995703dcd27935cf0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:06.237846 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7dc72e65-d6f7-4973-8ee5-dc0577e2dcf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '632f774bcff747e995703dcd27935cf0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:06.250035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7dc72e65-d6f7-4973-8ee5-dc0577e2dcf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '632f774bcff747e995703dcd27935cf0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:06.258617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7dc72e65-d6f7-4973-8ee5-dc0577e2dcf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '632f774bcff747e995703dcd27935cf0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:06.259597 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-7dc72e65-d6f7-4973-8ee5-dc0577e2dcf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '632f774bcff747e995703dcd27935cf0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:06.293658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 982/1988] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:06 2022] DELETE /identity/v3/users/632f774bcff747e995703dcd27935cf0 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:06.293848 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645ca82-52dc-4bae-a8c8-d12a266cd7d8 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:06.294333 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645ca82-52dc-4bae-a8c8-d12a266cd7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:06.294420 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645ca82-52dc-4bae-a8c8-d12a266cd7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:06.315827 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-d645ca82-52dc-4bae-a8c8-d12a266cd7d8 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:06.317544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d645ca82-52dc-4bae-a8c8-d12a266cd7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:06.333667 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:06.404926 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1007/1989] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:07:06.422933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc72e65-d6f7-4973-8ee5-dc0577e2dcf9 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:06.425556 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7dc72e65-d6f7-4973-8ee5-dc0577e2dcf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:06.540178 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc72e65-d6f7-4973-8ee5-dc0577e2dcf9 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:06.544119 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cebce-7e5d-424e-b7c7-91ec886d53b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:06.544814 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cebce-7e5d-424e-b7c7-91ec886d53b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:06.545090 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cebce-7e5d-424e-b7c7-91ec886d53b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:06.545366 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cebce-7e5d-424e-b7c7-91ec886d53b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:06.561427 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cebce-7e5d-424e-b7c7-91ec886d53b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:06.562994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cebce-7e5d-424e-b7c7-91ec886d53b4 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:06.574109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 983/1990] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:07.336835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645ca82-52dc-4bae-a8c8-d12a266cd7d8 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:07.340846 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d645ca82-52dc-4bae-a8c8-d12a266cd7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:07.363601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645ca82-52dc-4bae-a8c8-d12a266cd7d8 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:07.379166 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa256d-fd35-49d1-95f0-21ddcd11f4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:07.383258 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa256d-fd35-49d1-95f0-21ddcd11f4a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:07.383548 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa256d-fd35-49d1-95f0-21ddcd11f4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:07.383830 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa256d-fd35-49d1-95f0-21ddcd11f4a4 admin admin] PATH_INFO: `/v3/projects/5c26b071c98d4801bf1964bd53ea3b23` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:07.386474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa256d-fd35-49d1-95f0-21ddcd11f4a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c26b071c98d4801bf1964bd53ea3b23)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:07.393379 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa256d-fd35-49d1-95f0-21ddcd11f4a4 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:07.456734 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a9fa256d-fd35-49d1-95f0-21ddcd11f4a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c26b071c98d4801bf1964bd53ea3b23', 'request_id': 'req-a9fa256d-fd35-49d1-95f0-21ddcd11f4a4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:07.465695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a9fa256d-fd35-49d1-95f0-21ddcd11f4a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c26b071c98d4801bf1964bd53ea3b23', 'request_id': 'req-a9fa256d-fd35-49d1-95f0-21ddcd11f4a4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:07.474891 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a9fa256d-fd35-49d1-95f0-21ddcd11f4a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c26b071c98d4801bf1964bd53ea3b23', 'request_id': 'req-a9fa256d-fd35-49d1-95f0-21ddcd11f4a4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:07.492695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a9fa256d-fd35-49d1-95f0-21ddcd11f4a4 admin admin] The token cache is being invalidate because project 5c26b071c98d4801bf1964bd53ea3b23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:07.492980 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a9fa256d-fd35-49d1-95f0-21ddcd11f4a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:07.493871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a9fa256d-fd35-49d1-95f0-21ddcd11f4a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:07.495965 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1008/1991] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:07 2022] DELETE /identity/v3/projects/5c26b071c98d4801bf1964bd53ea3b23 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:07.502314 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed98976-95f4-44cd-9f10-1fd1b92847de None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:07.502807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed98976-95f4-44cd-9f10-1fd1b92847de None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:07.502908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed98976-95f4-44cd-9f10-1fd1b92847de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:07.524293 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-6ed98976-95f4-44cd-9f10-1fd1b92847de None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:07.525569 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6ed98976-95f4-44cd-9f10-1fd1b92847de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:07.624765 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 984/1992] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:07.632890 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7be3d-86fa-47de-9ada-2420dc71ac64 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:07.633148 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7be3d-86fa-47de-9ada-2420dc71ac64 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:07.633433 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7be3d-86fa-47de-9ada-2420dc71ac64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:07.666196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f0e7be3d-86fa-47de-9ada-2420dc71ac64 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:07.667302 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f0e7be3d-86fa-47de-9ada-2420dc71ac64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:07.680368 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1009/1993] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:07.685383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed98976-95f4-44cd-9f10-1fd1b92847de None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:07.703312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed98976-95f4-44cd-9f10-1fd1b92847de None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:07.711505 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436094d-71db-4c85-9d9b-5f449fde3b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:07.712650 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436094d-71db-4c85-9d9b-5f449fde3b19 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:07.713018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436094d-71db-4c85-9d9b-5f449fde3b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:07.713545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436094d-71db-4c85-9d9b-5f449fde3b19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:07.714501 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436094d-71db-4c85-9d9b-5f449fde3b19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:07:07.714962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436094d-71db-4c85-9d9b-5f449fde3b19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:07.717103 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436094d-71db-4c85-9d9b-5f449fde3b19 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:07.746652 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 985/1994] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:07:07.747595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7be3d-86fa-47de-9ada-2420dc71ac64 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:07.760849 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7be3d-86fa-47de-9ada-2420dc71ac64 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:07.766442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96e304-45c4-431e-95e8-a017d0b787e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:07.767216 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96e304-45c4-431e-95e8-a017d0b787e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:07.767581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96e304-45c4-431e-95e8-a017d0b787e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:07.767866 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96e304-45c4-431e-95e8-a017d0b787e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:07.768615 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96e304-45c4-431e-95e8-a017d0b787e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:07.770684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96e304-45c4-431e-95e8-a017d0b787e3 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:07.859678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1010/1995] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:07 2022] POST /identity/v3/projects => generated 397 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:07.863846 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436094d-71db-4c85-9d9b-5f449fde3b19 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:07.875546 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436094d-71db-4c85-9d9b-5f449fde3b19 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:07.887594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:07.950913 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08422e-d8c7-4598-96d4-b25cb28e79c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:07.951697 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08422e-d8c7-4598-96d4-b25cb28e79c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:07.952048 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08422e-d8c7-4598-96d4-b25cb28e79c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:07.952548 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08422e-d8c7-4598-96d4-b25cb28e79c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:07.953368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08422e-d8c7-4598-96d4-b25cb28e79c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:07.955381 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08422e-d8c7-4598-96d4-b25cb28e79c8 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:07.987882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 986/1996] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:07 2022] POST /identity/v3/users => generated 299 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:07.988899 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96e304-45c4-431e-95e8-a017d0b787e3 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:08.002120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96e304-45c4-431e-95e8-a017d0b787e3 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:08.016961 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc34bfb0-0729-4f67-a97b-970ea2a51dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:08.018256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34bfb0-0729-4f67-a97b-970ea2a51dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:08.019043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34bfb0-0729-4f67-a97b-970ea2a51dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:08.019621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34bfb0-0729-4f67-a97b-970ea2a51dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:08.020924 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc34bfb0-0729-4f67-a97b-970ea2a51dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:08.022788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc34bfb0-0729-4f67-a97b-970ea2a51dd9 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:08.037316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1011/1997] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:08.041385 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08422e-d8c7-4598-96d4-b25cb28e79c8 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:08.055157 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08422e-d8c7-4598-96d4-b25cb28e79c8 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:08.061928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b25b1-5122-41fe-b34f-e6295f04567f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:08.063700 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b25b1-5122-41fe-b34f-e6295f04567f admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:08.064234 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b25b1-5122-41fe-b34f-e6295f04567f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:08.064723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b25b1-5122-41fe-b34f-e6295f04567f admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:08.065686 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b25b1-5122-41fe-b34f-e6295f04567f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:08.072135 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b25b1-5122-41fe-b34f-e6295f04567f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:08.082011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 987/1998] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:07:08.087325 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc34bfb0-0729-4f67-a97b-970ea2a51dd9 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:08.104622 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc34bfb0-0729-4f67-a97b-970ea2a51dd9 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:08.110484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a6f4b-5731-46da-aedf-0cc7cf9cd8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:08.111130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a6f4b-5731-46da-aedf-0cc7cf9cd8ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:08.111396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a6f4b-5731-46da-aedf-0cc7cf9cd8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:08.112452 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a6f4b-5731-46da-aedf-0cc7cf9cd8ff admin admin] PATH_INFO: `/v3/projects/3ea4e3074a5143f4a2a2f02c217dd09e/users/935fc9b01b5b4fa9811450342bfb9240/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:08.135759 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a6f4b-5731-46da-aedf-0cc7cf9cd8ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ea4e3074a5143f4a2a2f02c217dd09e, user_id=935fc9b01b5b4fa9811450342bfb9240, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:08.139302 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a6f4b-5731-46da-aedf-0cc7cf9cd8ff admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:08.167965 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1012/1999] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:08 2022] PUT /identity/v3/projects/3ea4e3074a5143f4a2a2f02c217dd09e/users/935fc9b01b5b4fa9811450342bfb9240/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:08.172006 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3495fb-fdf6-4242-b53d-95118a0700b4 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:08.172943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3495fb-fdf6-4242-b53d-95118a0700b4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:08.173230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3495fb-fdf6-4242-b53d-95118a0700b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:08.205634 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ac3495fb-fdf6-4242-b53d-95118a0700b4 None None] MFA Rules not processed for user `935fc9b01b5b4fa9811450342bfb9240`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:08.206856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ac3495fb-fdf6-4242-b53d-95118a0700b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:08.300875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 988/2000] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:08.314229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a6f4b-5731-46da-aedf-0cc7cf9cd8ff admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:08.316471 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3c3a6f4b-5731-46da-aedf-0cc7cf9cd8ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:08.364700 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:08.388336 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a6f4b-5731-46da-aedf-0cc7cf9cd8ff admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:08.392615 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0f199-3695-47dc-bd5b-c3d3a45f8d59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:08.393213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0f199-3695-47dc-bd5b-c3d3a45f8d59 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:08.393504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0f199-3695-47dc-bd5b-c3d3a45f8d59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:08.393762 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0f199-3695-47dc-bd5b-c3d3a45f8d59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:08.405983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0f199-3695-47dc-bd5b-c3d3a45f8d59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:08.407505 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0f199-3695-47dc-bd5b-c3d3a45f8d59 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:08.417987 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1013/2001] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:08 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:15.669708 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3495fb-fdf6-4242-b53d-95118a0700b4 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:15.671533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ac3495fb-fdf6-4242-b53d-95118a0700b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:15.738306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3495fb-fdf6-4242-b53d-95118a0700b4 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:15.744389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28968c0b-de5c-4cea-8d33-b37e16709669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:15.745312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28968c0b-de5c-4cea-8d33-b37e16709669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:15.745590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28968c0b-de5c-4cea-8d33-b37e16709669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:15.745861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28968c0b-de5c-4cea-8d33-b37e16709669 admin admin] PATH_INFO: `/v3/users/8db0221ba50f4497b65bafdf37b9672b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:15.747188 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28968c0b-de5c-4cea-8d33-b37e16709669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8db0221ba50f4497b65bafdf37b9672b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:15.749198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28968c0b-de5c-4cea-8d33-b37e16709669 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:15.773481 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:15.801666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-28968c0b-de5c-4cea-8d33-b37e16709669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8db0221ba50f4497b65bafdf37b9672b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:15.802362 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-28968c0b-de5c-4cea-8d33-b37e16709669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8db0221ba50f4497b65bafdf37b9672b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:15.810031 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-28968c0b-de5c-4cea-8d33-b37e16709669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8db0221ba50f4497b65bafdf37b9672b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:15.817867 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-28968c0b-de5c-4cea-8d33-b37e16709669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8db0221ba50f4497b65bafdf37b9672b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:15.818605 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-28968c0b-de5c-4cea-8d33-b37e16709669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8db0221ba50f4497b65bafdf37b9672b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:15.837965 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 989/2002] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:15 2022] DELETE /identity/v3/users/8db0221ba50f4497b65bafdf37b9672b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:15.842949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabb0eb-4273-47ee-92e7-b08d0c395303 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:15.843470 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabb0eb-4273-47ee-92e7-b08d0c395303 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:15.843470 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fabb0eb-4273-47ee-92e7-b08d0c395303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:15.863480 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-4fabb0eb-4273-47ee-92e7-b08d0c395303 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:15.864748 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4fabb0eb-4273-47ee-92e7-b08d0c395303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:15.927433 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1014/2003] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:15.937392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28968c0b-de5c-4cea-8d33-b37e16709669 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:15.939125 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-28968c0b-de5c-4cea-8d33-b37e16709669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:16.002721 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28968c0b-de5c-4cea-8d33-b37e16709669 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:16.006821 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1487d7-7716-41ed-b244-b4b6ce1c1a10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:16.007470 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1487d7-7716-41ed-b244-b4b6ce1c1a10 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:16.007754 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1487d7-7716-41ed-b244-b4b6ce1c1a10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:16.008027 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1487d7-7716-41ed-b244-b4b6ce1c1a10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:16.018693 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1487d7-7716-41ed-b244-b4b6ce1c1a10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:16.020555 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1487d7-7716-41ed-b244-b4b6ce1c1a10 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:16.030391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 990/2004] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:16.312870 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabb0eb-4273-47ee-92e7-b08d0c395303 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:16.314278 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4fabb0eb-4273-47ee-92e7-b08d0c395303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:16.330578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fabb0eb-4273-47ee-92e7-b08d0c395303 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:16.335894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290df65-7ec8-480c-8090-8308f978befd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:16.337957 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290df65-7ec8-480c-8090-8308f978befd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:16.338591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290df65-7ec8-480c-8090-8308f978befd admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:16.338954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290df65-7ec8-480c-8090-8308f978befd admin admin] PATH_INFO: `/v3/projects/41cf47caff97420390fea1a6ed4ed6da` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:16.340320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290df65-7ec8-480c-8090-8308f978befd admin admin] RBAC: Authorizing `identity:delete_project(project_id=41cf47caff97420390fea1a6ed4ed6da)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:16.342451 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290df65-7ec8-480c-8090-8308f978befd admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:16.393737 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1290df65-7ec8-480c-8090-8308f978befd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41cf47caff97420390fea1a6ed4ed6da', 'request_id': 'req-1290df65-7ec8-480c-8090-8308f978befd'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:16.399555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1290df65-7ec8-480c-8090-8308f978befd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41cf47caff97420390fea1a6ed4ed6da', 'request_id': 'req-1290df65-7ec8-480c-8090-8308f978befd'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:16.406060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1290df65-7ec8-480c-8090-8308f978befd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41cf47caff97420390fea1a6ed4ed6da', 'request_id': 'req-1290df65-7ec8-480c-8090-8308f978befd'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:16.419935 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1290df65-7ec8-480c-8090-8308f978befd admin admin] The token cache is being invalidate because project 41cf47caff97420390fea1a6ed4ed6da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:16.420253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1290df65-7ec8-480c-8090-8308f978befd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:16.420890 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1290df65-7ec8-480c-8090-8308f978befd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:16.422444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1015/2005] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:16 2022] DELETE /identity/v3/projects/41cf47caff97420390fea1a6ed4ed6da => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:16.427620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1487d7-7716-41ed-b244-b4b6ce1c1a10 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:16.437685 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1487d7-7716-41ed-b244-b4b6ce1c1a10 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:16.515448 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf3f37-5de9-4406-a624-b9e4c5bbdda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:16.516352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf3f37-5de9-4406-a624-b9e4c5bbdda3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:16.516679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf3f37-5de9-4406-a624-b9e4c5bbdda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:16.517021 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf3f37-5de9-4406-a624-b9e4c5bbdda3 admin admin] PATH_INFO: `/v3/users/0cd971a0002f417b86c29d33ff3efa41` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:16.518358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf3f37-5de9-4406-a624-b9e4c5bbdda3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cd971a0002f417b86c29d33ff3efa41)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:16.520484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf3f37-5de9-4406-a624-b9e4c5bbdda3 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:16.582509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6aaf3f37-5de9-4406-a624-b9e4c5bbdda3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cd971a0002f417b86c29d33ff3efa41'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:16.583383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:16.584268 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6aaf3f37-5de9-4406-a624-b9e4c5bbdda3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cd971a0002f417b86c29d33ff3efa41'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:16.594796 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6aaf3f37-5de9-4406-a624-b9e4c5bbdda3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cd971a0002f417b86c29d33ff3efa41'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:16.608455 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6aaf3f37-5de9-4406-a624-b9e4c5bbdda3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cd971a0002f417b86c29d33ff3efa41'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:16.609683 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6aaf3f37-5de9-4406-a624-b9e4c5bbdda3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cd971a0002f417b86c29d33ff3efa41'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:16.634012 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 991/2006] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:16 2022] DELETE /identity/v3/users/0cd971a0002f417b86c29d33ff3efa41 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:17.100207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290df65-7ec8-480c-8090-8308f978befd admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:17.101918 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1290df65-7ec8-480c-8090-8308f978befd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:17.115459 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290df65-7ec8-480c-8090-8308f978befd admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:17.184587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbec14-401c-4340-90c0-cc72b8337257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:17.185445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbec14-401c-4340-90c0-cc72b8337257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:17.185737 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbec14-401c-4340-90c0-cc72b8337257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:17.185990 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbec14-401c-4340-90c0-cc72b8337257 admin admin] PATH_INFO: `/v3/projects/6cb8433b8306413c91268b71420d873c` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:17.187195 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbec14-401c-4340-90c0-cc72b8337257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cb8433b8306413c91268b71420d873c)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:17.189143 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbec14-401c-4340-90c0-cc72b8337257 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:17.195523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:17.236796 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-8afbec14-401c-4340-90c0-cc72b8337257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cb8433b8306413c91268b71420d873c', 'request_id': 'req-8afbec14-401c-4340-90c0-cc72b8337257'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:17.242780 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-8afbec14-401c-4340-90c0-cc72b8337257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cb8433b8306413c91268b71420d873c', 'request_id': 'req-8afbec14-401c-4340-90c0-cc72b8337257'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:17.249163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-8afbec14-401c-4340-90c0-cc72b8337257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cb8433b8306413c91268b71420d873c', 'request_id': 'req-8afbec14-401c-4340-90c0-cc72b8337257'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:17.263125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-8afbec14-401c-4340-90c0-cc72b8337257 admin admin] The token cache is being invalidate because project 6cb8433b8306413c91268b71420d873c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:17.263412 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-8afbec14-401c-4340-90c0-cc72b8337257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:17.264037 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-8afbec14-401c-4340-90c0-cc72b8337257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:17.265968 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1016/2007] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:17 2022] DELETE /identity/v3/projects/6cb8433b8306413c91268b71420d873c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:17.272062 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a81de-8939-4340-89cd-2d5e738a2f04 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:17.272604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a81de-8939-4340-89cd-2d5e738a2f04 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:17.272845 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a81de-8939-4340-89cd-2d5e738a2f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:17.299753 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-9b7a81de-8939-4340-89cd-2d5e738a2f04 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:17.301024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9b7a81de-8939-4340-89cd-2d5e738a2f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:17.369643 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 992/2008] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:17.377449 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046e655-43b0-4554-b70d-ae4e79a289cc None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:17.377736 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046e655-43b0-4554-b70d-ae4e79a289cc None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:17.378014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046e655-43b0-4554-b70d-ae4e79a289cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:17.398143 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8046e655-43b0-4554-b70d-ae4e79a289cc None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:17.399371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8046e655-43b0-4554-b70d-ae4e79a289cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:17.407975 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1017/2009] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:17.412214 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a81de-8939-4340-89cd-2d5e738a2f04 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:17.422710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a81de-8939-4340-89cd-2d5e738a2f04 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:17.427572 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e1c72-c53b-4be4-bd18-4d6139db4270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:17.428716 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e1c72-c53b-4be4-bd18-4d6139db4270 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:17.428716 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e1c72-c53b-4be4-bd18-4d6139db4270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:17.428986 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e1c72-c53b-4be4-bd18-4d6139db4270 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:17.429809 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e1c72-c53b-4be4-bd18-4d6139db4270 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:07:17.430109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e1c72-c53b-4be4-bd18-4d6139db4270 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:17.431690 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e1c72-c53b-4be4-bd18-4d6139db4270 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:17.443806 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 993/2010] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:07:17.448969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046e655-43b0-4554-b70d-ae4e79a289cc None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:17.466907 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046e655-43b0-4554-b70d-ae4e79a289cc None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:17.471693 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901e783-c58d-4026-b918-81806fc1de1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:17.472347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901e783-c58d-4026-b918-81806fc1de1b admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:17.472655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901e783-c58d-4026-b918-81806fc1de1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:17.472924 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901e783-c58d-4026-b918-81806fc1de1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:17.473607 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901e783-c58d-4026-b918-81806fc1de1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:17.475834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e901e783-c58d-4026-b918-81806fc1de1b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:17.507371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1018/2011] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:17 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:17.511161 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e1c72-c53b-4be4-bd18-4d6139db4270 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:17.523597 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e1c72-c53b-4be4-bd18-4d6139db4270 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:17.588824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e24b79-21d6-4927-966e-3579ae1b925b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:17.589492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e24b79-21d6-4927-966e-3579ae1b925b admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:17.589771 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e24b79-21d6-4927-966e-3579ae1b925b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:17.590040 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e24b79-21d6-4927-966e-3579ae1b925b admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:17.590723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e24b79-21d6-4927-966e-3579ae1b925b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:17.592672 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e24b79-21d6-4927-966e-3579ae1b925b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:17.619214 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 994/2012] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:17 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:17.625551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901e783-c58d-4026-b918-81806fc1de1b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:17.638319 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901e783-c58d-4026-b918-81806fc1de1b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:17.643520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0667279-3606-4b61-b34f-7dbae2d79cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:17.644184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0667279-3606-4b61-b34f-7dbae2d79cdf admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:17.644486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0667279-3606-4b61-b34f-7dbae2d79cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:17.644761 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0667279-3606-4b61-b34f-7dbae2d79cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:17.645485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0667279-3606-4b61-b34f-7dbae2d79cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:17.647099 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0667279-3606-4b61-b34f-7dbae2d79cdf admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:17.658066 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1019/2013] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:17.664238 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e24b79-21d6-4927-966e-3579ae1b925b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:17.676898 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e24b79-21d6-4927-966e-3579ae1b925b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:17.683850 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76195e16-856b-4ac3-b167-1bf4367016ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:17.684573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76195e16-856b-4ac3-b167-1bf4367016ff admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:17.684852 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76195e16-856b-4ac3-b167-1bf4367016ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:17.685108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76195e16-856b-4ac3-b167-1bf4367016ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:17.685839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76195e16-856b-4ac3-b167-1bf4367016ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:17.687459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76195e16-856b-4ac3-b167-1bf4367016ff admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:17.705102 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 995/2014] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:17.706394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0667279-3606-4b61-b34f-7dbae2d79cdf admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:17.717243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0667279-3606-4b61-b34f-7dbae2d79cdf admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:17.723546 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b036c6d-a05d-445f-af0e-fc3e8acb43a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:17.724948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b036c6d-a05d-445f-af0e-fc3e8acb43a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:17.724948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b036c6d-a05d-445f-af0e-fc3e8acb43a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:17.725159 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b036c6d-a05d-445f-af0e-fc3e8acb43a8 admin admin] PATH_INFO: `/v3/projects/658ef933ba61420da8cd9a54c3ec7bc7/users/3734082fdda041d281318bb7997da87d/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:17.752816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b036c6d-a05d-445f-af0e-fc3e8acb43a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=658ef933ba61420da8cd9a54c3ec7bc7, user_id=3734082fdda041d281318bb7997da87d, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:17.755638 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b036c6d-a05d-445f-af0e-fc3e8acb43a8 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:17.792507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1020/2015] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:17 2022] PUT /identity/v3/projects/658ef933ba61420da8cd9a54c3ec7bc7/users/3734082fdda041d281318bb7997da87d/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:17.794476 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d25405-79ad-43a8-bd24-62b211325859 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:17.794763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d25405-79ad-43a8-bd24-62b211325859 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:17.795033 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d25405-79ad-43a8-bd24-62b211325859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:17.827242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-30d25405-79ad-43a8-bd24-62b211325859 None None] MFA Rules not processed for user `3734082fdda041d281318bb7997da87d`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:17.828493 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-30d25405-79ad-43a8-bd24-62b211325859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:17.853518 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:17.918691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 996/2016] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:17.923470 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b036c6d-a05d-445f-af0e-fc3e8acb43a8 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:17.991187 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b036c6d-a05d-445f-af0e-fc3e8acb43a8 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:17.996719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305b5ba-379f-4834-8a84-35ec306b434b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:17.997406 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305b5ba-379f-4834-8a84-35ec306b434b admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:17.997686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305b5ba-379f-4834-8a84-35ec306b434b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:17.997960 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305b5ba-379f-4834-8a84-35ec306b434b admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:17.998723 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305b5ba-379f-4834-8a84-35ec306b434b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:18.000597 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305b5ba-379f-4834-8a84-35ec306b434b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:18.033395 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1021/2017] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:17 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:18.037520 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d25405-79ad-43a8-bd24-62b211325859 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:18.049479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d25405-79ad-43a8-bd24-62b211325859 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:18.124266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0eac9-d20c-4d2a-87c8-2a8b302ddf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:18.124973 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0eac9-d20c-4d2a-87c8-2a8b302ddf9e admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:18.125257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0eac9-d20c-4d2a-87c8-2a8b302ddf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:18.125551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0eac9-d20c-4d2a-87c8-2a8b302ddf9e admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:18.126239 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0eac9-d20c-4d2a-87c8-2a8b302ddf9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:18.128162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0eac9-d20c-4d2a-87c8-2a8b302ddf9e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:18.151804 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 997/2018] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:18 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:07:18.153351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305b5ba-379f-4834-8a84-35ec306b434b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:18.165909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305b5ba-379f-4834-8a84-35ec306b434b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:18.171652 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980f511-7431-4fa1-85a4-2791c6bbde0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:18.172379 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980f511-7431-4fa1-85a4-2791c6bbde0e admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:18.172784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980f511-7431-4fa1-85a4-2791c6bbde0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:18.173025 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980f511-7431-4fa1-85a4-2791c6bbde0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:18.173923 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980f511-7431-4fa1-85a4-2791c6bbde0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:18.175659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980f511-7431-4fa1-85a4-2791c6bbde0e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:18.188264 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1022/2019] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:18.192600 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0eac9-d20c-4d2a-87c8-2a8b302ddf9e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:18.203244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0eac9-d20c-4d2a-87c8-2a8b302ddf9e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:18.208441 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ee5ea-593b-4d1e-b704-a4ee82fa95c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:18.209105 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ee5ea-593b-4d1e-b704-a4ee82fa95c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:18.209439 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ee5ea-593b-4d1e-b704-a4ee82fa95c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:18.209719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ee5ea-593b-4d1e-b704-a4ee82fa95c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:18.210442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ee5ea-593b-4d1e-b704-a4ee82fa95c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:18.212054 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ee5ea-593b-4d1e-b704-a4ee82fa95c3 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:18.223070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 998/2020] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:18.227684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980f511-7431-4fa1-85a4-2791c6bbde0e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:18.239447 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:18.243793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980f511-7431-4fa1-85a4-2791c6bbde0e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:18.250996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ff9d7-0a67-478c-9ad6-61d3e0e0f2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:18.252243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ff9d7-0a67-478c-9ad6-61d3e0e0f2be admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:18.252701 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ff9d7-0a67-478c-9ad6-61d3e0e0f2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:18.253056 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ff9d7-0a67-478c-9ad6-61d3e0e0f2be admin admin] PATH_INFO: `/v3/projects/f74f5b14b747445a8302ccae24e3afd5/users/bcf152e117da4f618dd4321b3691e7a7/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:18.277154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ff9d7-0a67-478c-9ad6-61d3e0e0f2be admin admin] RBAC: Authorizing `identity:create_grant(project_id=f74f5b14b747445a8302ccae24e3afd5, user_id=bcf152e117da4f618dd4321b3691e7a7, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:18.281298 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7ff9d7-0a67-478c-9ad6-61d3e0e0f2be admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:18.301312 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1023/2021] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:18 2022] PUT /identity/v3/projects/f74f5b14b747445a8302ccae24e3afd5/users/bcf152e117da4f618dd4321b3691e7a7/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:18.311956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58abee7-88f5-4472-9f73-5af6a198e8ee None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:18.312457 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58abee7-88f5-4472-9f73-5af6a198e8ee None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:18.312580 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58abee7-88f5-4472-9f73-5af6a198e8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:18.341893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a58abee7-88f5-4472-9f73-5af6a198e8ee None None] MFA Rules not processed for user `bcf152e117da4f618dd4321b3691e7a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:18.343125 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a58abee7-88f5-4472-9f73-5af6a198e8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:18.411051 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 999/2022] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:18.416131 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ff9d7-0a67-478c-9ad6-61d3e0e0f2be admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:18.489884 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ff9d7-0a67-478c-9ad6-61d3e0e0f2be admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:18.495123 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61591fd-c9da-4a96-9f94-93c477dbc984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:18.496032 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61591fd-c9da-4a96-9f94-93c477dbc984 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:18.496523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61591fd-c9da-4a96-9f94-93c477dbc984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:18.496904 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61591fd-c9da-4a96-9f94-93c477dbc984 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:18.497846 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61591fd-c9da-4a96-9f94-93c477dbc984 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:18.501301 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61591fd-c9da-4a96-9f94-93c477dbc984 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:18.527832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1024/2023] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:18 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:18.531826 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58abee7-88f5-4472-9f73-5af6a198e8ee None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:18.543863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58abee7-88f5-4472-9f73-5af6a198e8ee None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:18.570333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:18.609013 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ad8b8-9643-4fab-b27e-64b6329fbee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:18.609780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ad8b8-9643-4fab-b27e-64b6329fbee2 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:18.609967 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ad8b8-9643-4fab-b27e-64b6329fbee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:18.610237 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ad8b8-9643-4fab-b27e-64b6329fbee2 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:18.611001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6ad8b8-9643-4fab-b27e-64b6329fbee2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:18.613022 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6ad8b8-9643-4fab-b27e-64b6329fbee2 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:18.637838 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1000/2024] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:18 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:18.642476 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61591fd-c9da-4a96-9f94-93c477dbc984 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:18.654005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61591fd-c9da-4a96-9f94-93c477dbc984 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:18.659130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9644110-dc2f-46fc-9ef6-abf62884319e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:18.659833 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9644110-dc2f-46fc-9ef6-abf62884319e admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:18.660080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9644110-dc2f-46fc-9ef6-abf62884319e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:18.660593 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9644110-dc2f-46fc-9ef6-abf62884319e admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:18.661328 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9644110-dc2f-46fc-9ef6-abf62884319e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:18.663280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9644110-dc2f-46fc-9ef6-abf62884319e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:18.674799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1025/2025] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:18.680600 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ad8b8-9643-4fab-b27e-64b6329fbee2 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:18.691057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ad8b8-9643-4fab-b27e-64b6329fbee2 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:18.696765 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c6bf4-1df3-454b-b5d1-e1818d3b02cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:18.697655 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c6bf4-1df3-454b-b5d1-e1818d3b02cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:18.697941 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c6bf4-1df3-454b-b5d1-e1818d3b02cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:18.698215 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c6bf4-1df3-454b-b5d1-e1818d3b02cc admin admin] PATH_INFO: `/v3/projects/a98f540a420f4561980508bd762ebd52/users/dc131acf310040e19fe59c36fd23f413/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:18.714035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c6bf4-1df3-454b-b5d1-e1818d3b02cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a98f540a420f4561980508bd762ebd52, user_id=dc131acf310040e19fe59c36fd23f413, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:18.716740 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c6bf4-1df3-454b-b5d1-e1818d3b02cc admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:18.731437 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1001/2026] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:18 2022] PUT /identity/v3/projects/a98f540a420f4561980508bd762ebd52/users/dc131acf310040e19fe59c36fd23f413/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:07:18.735845 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9644110-dc2f-46fc-9ef6-abf62884319e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:18.746706 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9644110-dc2f-46fc-9ef6-abf62884319e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:18.811501 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd92235-7083-4b85-a8e8-e12cabf24b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:18.812208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd92235-7083-4b85-a8e8-e12cabf24b48 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:18.812611 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd92235-7083-4b85-a8e8-e12cabf24b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:18.812897 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd92235-7083-4b85-a8e8-e12cabf24b48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:18.813658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd92235-7083-4b85-a8e8-e12cabf24b48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:18.815383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd92235-7083-4b85-a8e8-e12cabf24b48 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:18.826698 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1026/2027] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:18.831319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c6bf4-1df3-454b-b5d1-e1818d3b02cc admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:18.852244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c6bf4-1df3-454b-b5d1-e1818d3b02cc admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:18.858026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb7328-1f74-401e-9cf0-e350f80afdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:18.859353 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb7328-1f74-401e-9cf0-e350f80afdb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:18.859353 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb7328-1f74-401e-9cf0-e350f80afdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:18.859554 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb7328-1f74-401e-9cf0-e350f80afdb9 admin admin] PATH_INFO: `/v3/projects/a98f540a420f4561980508bd762ebd52/users/dc131acf310040e19fe59c36fd23f413/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:18.861675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb7328-1f74-401e-9cf0-e350f80afdb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a98f540a420f4561980508bd762ebd52, user_id=dc131acf310040e19fe59c36fd23f413, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:18.864510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb7328-1f74-401e-9cf0-e350f80afdb9 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:18.876483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1002/2028] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:18 2022] PUT /identity/v3/projects/a98f540a420f4561980508bd762ebd52/users/dc131acf310040e19fe59c36fd23f413/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:18.888782 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb554ca-8edb-49ce-a81b-996d05578dee None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:18.889106 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb554ca-8edb-49ce-a81b-996d05578dee None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:18.889404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb554ca-8edb-49ce-a81b-996d05578dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:18.919258 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8fb554ca-8edb-49ce-a81b-996d05578dee None None] MFA Rules not processed for user `dc131acf310040e19fe59c36fd23f413`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:18.920687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8fb554ca-8edb-49ce-a81b-996d05578dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:18.968237 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:19.002683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1027/2029] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:19.015589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb7328-1f74-401e-9cf0-e350f80afdb9 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:19.018169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-01fb7328-1f74-401e-9cf0-e350f80afdb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:19.080258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb7328-1f74-401e-9cf0-e350f80afdb9 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:19.084165 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b2e6d-a9c7-4d8f-8d33-d97f452c6f0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:19.085331 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b2e6d-a9c7-4d8f-8d33-d97f452c6f0e service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:19.085331 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b2e6d-a9c7-4d8f-8d33-d97f452c6f0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:19.085565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b2e6d-a9c7-4d8f-8d33-d97f452c6f0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:19.096286 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b2e6d-a9c7-4d8f-8d33-d97f452c6f0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:19.097761 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b2e6d-a9c7-4d8f-8d33-d97f452c6f0e service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:19.107600 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1003/2030] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:19.158702 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb554ca-8edb-49ce-a81b-996d05578dee None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:19.160836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8fb554ca-8edb-49ce-a81b-996d05578dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:19.175400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb554ca-8edb-49ce-a81b-996d05578dee None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:19.180087 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fa8a9-ec01-4561-aaf0-fa575f5001ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:19.180770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fa8a9-ec01-4561-aaf0-fa575f5001ac service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:19.181049 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fa8a9-ec01-4561-aaf0-fa575f5001ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:19.181319 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fa8a9-ec01-4561-aaf0-fa575f5001ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:19.241278 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325fa8a9-ec01-4561-aaf0-fa575f5001ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:19.242726 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325fa8a9-ec01-4561-aaf0-fa575f5001ac service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:19.252061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1028/2031] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:20.183252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b2e6d-a9c7-4d8f-8d33-d97f452c6f0e service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:20.184678 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6e4b2e6d-a9c7-4d8f-8d33-d97f452c6f0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:20.258535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b2e6d-a9c7-4d8f-8d33-d97f452c6f0e service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:20.263930 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf2f0f-dbb2-4aed-a0a4-99cc589d0447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:20.264892 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf2f0f-dbb2-4aed-a0a4-99cc589d0447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:20.265895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf2f0f-dbb2-4aed-a0a4-99cc589d0447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:20.265895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf2f0f-dbb2-4aed-a0a4-99cc589d0447 admin admin] PATH_INFO: `/v3/users/7b5127b7d5bc4a58afa9930108832697` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:20.266983 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf2f0f-dbb2-4aed-a0a4-99cc589d0447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b5127b7d5bc4a58afa9930108832697)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:20.268956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf2f0f-dbb2-4aed-a0a4-99cc589d0447 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:20.295455 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:20.325325 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6ccf2f0f-dbb2-4aed-a0a4-99cc589d0447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b5127b7d5bc4a58afa9930108832697'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:20.326178 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6ccf2f0f-dbb2-4aed-a0a4-99cc589d0447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b5127b7d5bc4a58afa9930108832697'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:20.335040 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6ccf2f0f-dbb2-4aed-a0a4-99cc589d0447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b5127b7d5bc4a58afa9930108832697'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:20.345095 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6ccf2f0f-dbb2-4aed-a0a4-99cc589d0447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b5127b7d5bc4a58afa9930108832697'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:20.346103 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6ccf2f0f-dbb2-4aed-a0a4-99cc589d0447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b5127b7d5bc4a58afa9930108832697'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:20.370824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1004/2032] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:20 2022] DELETE /identity/v3/users/7b5127b7d5bc4a58afa9930108832697 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:20.376462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617b584-9c7b-452d-8d74-93e16daf7c26 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:20.376949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617b584-9c7b-452d-8d74-93e16daf7c26 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:20.377024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617b584-9c7b-452d-8d74-93e16daf7c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:20.394671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e617b584-9c7b-452d-8d74-93e16daf7c26 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:20.395973 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e617b584-9c7b-452d-8d74-93e16daf7c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:20.459079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1029/2033] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:20.470572 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf2f0f-dbb2-4aed-a0a4-99cc589d0447 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:20.472216 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6ccf2f0f-dbb2-4aed-a0a4-99cc589d0447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:20.542073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf2f0f-dbb2-4aed-a0a4-99cc589d0447 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:20.556339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122063c6-6e90-4347-971b-b9c511f265a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:20.557066 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122063c6-6e90-4347-971b-b9c511f265a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:20.557360 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122063c6-6e90-4347-971b-b9c511f265a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:20.557668 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122063c6-6e90-4347-971b-b9c511f265a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:20.573131 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122063c6-6e90-4347-971b-b9c511f265a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:20.574623 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122063c6-6e90-4347-971b-b9c511f265a1 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:20.593093 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1005/2034] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:20.835171 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617b584-9c7b-452d-8d74-93e16daf7c26 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:20.837987 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e617b584-9c7b-452d-8d74-93e16daf7c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:20.873260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617b584-9c7b-452d-8d74-93e16daf7c26 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:20.883465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe9132-b076-434e-beb1-c44ac49b4534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:20.884830 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe9132-b076-434e-beb1-c44ac49b4534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:20.885475 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe9132-b076-434e-beb1-c44ac49b4534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:20.885996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe9132-b076-434e-beb1-c44ac49b4534 admin admin] PATH_INFO: `/v3/users/935fc9b01b5b4fa9811450342bfb9240` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:20.887610 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe9132-b076-434e-beb1-c44ac49b4534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=935fc9b01b5b4fa9811450342bfb9240)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:20.890563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe9132-b076-434e-beb1-c44ac49b4534 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:20.955412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122063c6-6e90-4347-971b-b9c511f265a1 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:20.956900 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-122063c6-6e90-4347-971b-b9c511f265a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:20.976892 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122063c6-6e90-4347-971b-b9c511f265a1 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:20.983771 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7f6a9-1ecd-444d-9544-eaf067d1d71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:20.984752 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7f6a9-1ecd-444d-9544-eaf067d1d71b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:20.985355 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7f6a9-1ecd-444d-9544-eaf067d1d71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:20.985639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7f6a9-1ecd-444d-9544-eaf067d1d71b admin admin] PATH_INFO: `/v3/projects/5cccd72e8918444c94a925e052a09fb2` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:20.988194 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7f6a9-1ecd-444d-9544-eaf067d1d71b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cccd72e8918444c94a925e052a09fb2)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:20.990658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7f6a9-1ecd-444d-9544-eaf067d1d71b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:21.007649 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-32fe9132-b076-434e-beb1-c44ac49b4534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '935fc9b01b5b4fa9811450342bfb9240'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.020585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-32fe9132-b076-434e-beb1-c44ac49b4534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '935fc9b01b5b4fa9811450342bfb9240'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.021316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-32fe9132-b076-434e-beb1-c44ac49b4534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '935fc9b01b5b4fa9811450342bfb9240'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.036923 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-32fe9132-b076-434e-beb1-c44ac49b4534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '935fc9b01b5b4fa9811450342bfb9240'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.040880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-32fe9132-b076-434e-beb1-c44ac49b4534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '935fc9b01b5b4fa9811450342bfb9240'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.048929 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:21.068881 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-14e7f6a9-1ecd-444d-9544-eaf067d1d71b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cccd72e8918444c94a925e052a09fb2', 'request_id': 'req-14e7f6a9-1ecd-444d-9544-eaf067d1d71b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.074681 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1030/2035] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:20 2022] DELETE /identity/v3/users/935fc9b01b5b4fa9811450342bfb9240 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:21.079373 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3092fe-2805-4923-acb9-ba0df81fb061 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:21.080045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-14e7f6a9-1ecd-444d-9544-eaf067d1d71b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cccd72e8918444c94a925e052a09fb2', 'request_id': 'req-14e7f6a9-1ecd-444d-9544-eaf067d1d71b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.082511 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3092fe-2805-4923-acb9-ba0df81fb061 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:21.082825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3092fe-2805-4923-acb9-ba0df81fb061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:21.088442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-14e7f6a9-1ecd-444d-9544-eaf067d1d71b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cccd72e8918444c94a925e052a09fb2', 'request_id': 'req-14e7f6a9-1ecd-444d-9544-eaf067d1d71b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.108954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7e3092fe-2805-4923-acb9-ba0df81fb061 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:21.110549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7e3092fe-2805-4923-acb9-ba0df81fb061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:21.115551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-14e7f6a9-1ecd-444d-9544-eaf067d1d71b admin admin] The token cache is being invalidate because project 5cccd72e8918444c94a925e052a09fb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:21.116401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-14e7f6a9-1ecd-444d-9544-eaf067d1d71b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.117094 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-14e7f6a9-1ecd-444d-9544-eaf067d1d71b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.119000 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1006/2036] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:20 2022] DELETE /identity/v3/projects/5cccd72e8918444c94a925e052a09fb2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:21.123374 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7f6a9-1ecd-444d-9544-eaf067d1d71b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:21.125777 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-14e7f6a9-1ecd-444d-9544-eaf067d1d71b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:21.177704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:21.194105 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1031/2037] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:21.209336 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3092fe-2805-4923-acb9-ba0df81fb061 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:21.211042 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7e3092fe-2805-4923-acb9-ba0df81fb061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:21.227185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7f6a9-1ecd-444d-9544-eaf067d1d71b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:21.232798 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06344698-d688-4dd0-b9c7-dd5c9e9e5e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:21.233897 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06344698-d688-4dd0-b9c7-dd5c9e9e5e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:21.234241 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06344698-d688-4dd0-b9c7-dd5c9e9e5e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:21.235074 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06344698-d688-4dd0-b9c7-dd5c9e9e5e82 admin admin] PATH_INFO: `/v3/users/d194d57b952441c2a55d6c8f7e9d709f` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:21.236025 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06344698-d688-4dd0-b9c7-dd5c9e9e5e82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d194d57b952441c2a55d6c8f7e9d709f)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:21.238697 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06344698-d688-4dd0-b9c7-dd5c9e9e5e82 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:21.291755 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3092fe-2805-4923-acb9-ba0df81fb061 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:21.298566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b18af6-a07d-4fda-9e96-4342abec5d50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:21.299505 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b18af6-a07d-4fda-9e96-4342abec5d50 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:21.299795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b18af6-a07d-4fda-9e96-4342abec5d50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:21.300081 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b18af6-a07d-4fda-9e96-4342abec5d50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:21.301870 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-06344698-d688-4dd0-b9c7-dd5c9e9e5e82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd194d57b952441c2a55d6c8f7e9d709f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.302602 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-06344698-d688-4dd0-b9c7-dd5c9e9e5e82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd194d57b952441c2a55d6c8f7e9d709f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.310914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b18af6-a07d-4fda-9e96-4342abec5d50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:21.312684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b18af6-a07d-4fda-9e96-4342abec5d50 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:21.313715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-06344698-d688-4dd0-b9c7-dd5c9e9e5e82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd194d57b952441c2a55d6c8f7e9d709f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.322001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-06344698-d688-4dd0-b9c7-dd5c9e9e5e82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd194d57b952441c2a55d6c8f7e9d709f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.322949 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-06344698-d688-4dd0-b9c7-dd5c9e9e5e82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd194d57b952441c2a55d6c8f7e9d709f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.325785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1032/2038] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:21.352198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1007/2039] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:21 2022] DELETE /identity/v3/users/d194d57b952441c2a55d6c8f7e9d709f => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:21.661150 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b18af6-a07d-4fda-9e96-4342abec5d50 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:21.662976 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-98b18af6-a07d-4fda-9e96-4342abec5d50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:21.683921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b18af6-a07d-4fda-9e96-4342abec5d50 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:21.729517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06344698-d688-4dd0-b9c7-dd5c9e9e5e82 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:21.731320 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-06344698-d688-4dd0-b9c7-dd5c9e9e5e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:21.751324 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d641988-65ab-453f-8574-0783db4b1057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:21.752458 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d641988-65ab-453f-8574-0783db4b1057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:21.752858 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d641988-65ab-453f-8574-0783db4b1057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:21.753019 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d641988-65ab-453f-8574-0783db4b1057 admin admin] PATH_INFO: `/v3/projects/3ea4e3074a5143f4a2a2f02c217dd09e` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:21.754260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d641988-65ab-453f-8574-0783db4b1057 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ea4e3074a5143f4a2a2f02c217dd09e)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:21.756258 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d641988-65ab-453f-8574-0783db4b1057 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:21.794775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:21.808944 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7d641988-65ab-453f-8574-0783db4b1057 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ea4e3074a5143f4a2a2f02c217dd09e', 'request_id': 'req-7d641988-65ab-453f-8574-0783db4b1057'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.813013 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06344698-d688-4dd0-b9c7-dd5c9e9e5e82 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:21.814838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7d641988-65ab-453f-8574-0783db4b1057 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ea4e3074a5143f4a2a2f02c217dd09e', 'request_id': 'req-7d641988-65ab-453f-8574-0783db4b1057'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.821317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7d641988-65ab-453f-8574-0783db4b1057 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ea4e3074a5143f4a2a2f02c217dd09e', 'request_id': 'req-7d641988-65ab-453f-8574-0783db4b1057'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.835146 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7d641988-65ab-453f-8574-0783db4b1057 admin admin] The token cache is being invalidate because project 3ea4e3074a5143f4a2a2f02c217dd09e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:21.835633 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7d641988-65ab-453f-8574-0783db4b1057 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.836114 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7d641988-65ab-453f-8574-0783db4b1057 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.838024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1033/2040] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:21 2022] DELETE /identity/v3/projects/3ea4e3074a5143f4a2a2f02c217dd09e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:21.872971 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:21.885789 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dcbb66-3ace-4a1d-9b1a-cf58f9836783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:21.887148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcbb66-3ace-4a1d-9b1a-cf58f9836783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:21.887662 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcbb66-3ace-4a1d-9b1a-cf58f9836783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:21.887942 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcbb66-3ace-4a1d-9b1a-cf58f9836783 admin admin] PATH_INFO: `/v3/projects/9e2c73ce49c941138e1c17c2f5ef07ee` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:21.889424 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dcbb66-3ace-4a1d-9b1a-cf58f9836783 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e2c73ce49c941138e1c17c2f5ef07ee)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:21.891777 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dcbb66-3ace-4a1d-9b1a-cf58f9836783 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:21.944689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a1dcbb66-3ace-4a1d-9b1a-cf58f9836783 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e2c73ce49c941138e1c17c2f5ef07ee', 'request_id': 'req-a1dcbb66-3ace-4a1d-9b1a-cf58f9836783'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.952724 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a1dcbb66-3ace-4a1d-9b1a-cf58f9836783 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e2c73ce49c941138e1c17c2f5ef07ee', 'request_id': 'req-a1dcbb66-3ace-4a1d-9b1a-cf58f9836783'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.960068 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a1dcbb66-3ace-4a1d-9b1a-cf58f9836783 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e2c73ce49c941138e1c17c2f5ef07ee', 'request_id': 'req-a1dcbb66-3ace-4a1d-9b1a-cf58f9836783'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.974523 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a1dcbb66-3ace-4a1d-9b1a-cf58f9836783 admin admin] The token cache is being invalidate because project 9e2c73ce49c941138e1c17c2f5ef07ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:21.974840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a1dcbb66-3ace-4a1d-9b1a-cf58f9836783 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.975543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a1dcbb66-3ace-4a1d-9b1a-cf58f9836783 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:21.977289 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1008/2041] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:21 2022] DELETE /identity/v3/projects/9e2c73ce49c941138e1c17c2f5ef07ee => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:07:21.982750 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08da6b0-19d5-4a33-9dc3-24ce170532cf None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:21.983040 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08da6b0-19d5-4a33-9dc3-24ce170532cf None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:21.983316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08da6b0-19d5-4a33-9dc3-24ce170532cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:22.003773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f08da6b0-19d5-4a33-9dc3-24ce170532cf None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:22.005037 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f08da6b0-19d5-4a33-9dc3-24ce170532cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:22.069794 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1034/2042] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:22.079115 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3ab20-b92f-4c7c-b471-51e68b21a70f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:22.079397 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3ab20-b92f-4c7c-b471-51e68b21a70f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:22.079670 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3ab20-b92f-4c7c-b471-51e68b21a70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:22.099779 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d2b3ab20-b92f-4c7c-b471-51e68b21a70f None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:22.101082 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d2b3ab20-b92f-4c7c-b471-51e68b21a70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:22.109546 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1009/2043] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:22.113866 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08da6b0-19d5-4a33-9dc3-24ce170532cf None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:22.126311 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08da6b0-19d5-4a33-9dc3-24ce170532cf None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:22.132191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276defa2-85a3-4df5-8838-47bf219dbb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:22.132922 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276defa2-85a3-4df5-8838-47bf219dbb11 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:22.133202 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276defa2-85a3-4df5-8838-47bf219dbb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:22.133475 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276defa2-85a3-4df5-8838-47bf219dbb11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:22.134241 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276defa2-85a3-4df5-8838-47bf219dbb11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:07:22.134549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276defa2-85a3-4df5-8838-47bf219dbb11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:22.136152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276defa2-85a3-4df5-8838-47bf219dbb11 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:22.149878 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1035/2044] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:07:22.154927 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3ab20-b92f-4c7c-b471-51e68b21a70f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:22.165455 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3ab20-b92f-4c7c-b471-51e68b21a70f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:22.171021 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7784d9-6086-40c8-a219-a224bcd481ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:22.171719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7784d9-6086-40c8-a219-a224bcd481ef admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:22.172014 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7784d9-6086-40c8-a219-a224bcd481ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:22.172311 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7784d9-6086-40c8-a219-a224bcd481ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:22.173067 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7784d9-6086-40c8-a219-a224bcd481ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:22.175381 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7784d9-6086-40c8-a219-a224bcd481ef admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:22.202529 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1010/2045] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:22 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:22.206838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276defa2-85a3-4df5-8838-47bf219dbb11 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:22.217358 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276defa2-85a3-4df5-8838-47bf219dbb11 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:22.290820 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871fe2f-5013-4f12-8cfe-868e3eef4583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:22.291495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871fe2f-5013-4f12-8cfe-868e3eef4583 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:22.291780 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871fe2f-5013-4f12-8cfe-868e3eef4583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:22.292076 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871fe2f-5013-4f12-8cfe-868e3eef4583 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:22.292812 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871fe2f-5013-4f12-8cfe-868e3eef4583 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:22.294730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871fe2f-5013-4f12-8cfe-868e3eef4583 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:22.319063 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1036/2046] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:22 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:22.321510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7784d9-6086-40c8-a219-a224bcd481ef admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:22.332751 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7784d9-6086-40c8-a219-a224bcd481ef admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:22.338610 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4cf3b-dc7a-491d-8f00-595554f69680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:22.339286 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4cf3b-dc7a-491d-8f00-595554f69680 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:22.339564 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4cf3b-dc7a-491d-8f00-595554f69680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:22.339833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4cf3b-dc7a-491d-8f00-595554f69680 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:22.340622 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d4cf3b-dc7a-491d-8f00-595554f69680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:22.342228 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d4cf3b-dc7a-491d-8f00-595554f69680 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:22.356468 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1011/2047] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:22.357817 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871fe2f-5013-4f12-8cfe-868e3eef4583 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:22.368086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871fe2f-5013-4f12-8cfe-868e3eef4583 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:22.373981 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3d623-7d9c-4bef-9072-3840631edfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:22.374432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3d623-7d9c-4bef-9072-3840631edfd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:22.374784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3d623-7d9c-4bef-9072-3840631edfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:22.375064 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3d623-7d9c-4bef-9072-3840631edfd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:22.375796 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3d623-7d9c-4bef-9072-3840631edfd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:22.381023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3d623-7d9c-4bef-9072-3840631edfd0 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:22.393329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1037/2048] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:22.397713 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4cf3b-dc7a-491d-8f00-595554f69680 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:22.399536 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3d623-7d9c-4bef-9072-3840631edfd0 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:22.402032 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b5d3d623-7d9c-4bef-9072-3840631edfd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:22.408117 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4cf3b-dc7a-491d-8f00-595554f69680 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:22.413672 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6053b5e-76d1-44e5-bc80-60e3f2fb378c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:22.414369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6053b5e-76d1-44e5-bc80-60e3f2fb378c admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:22.414658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6053b5e-76d1-44e5-bc80-60e3f2fb378c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:22.414935 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6053b5e-76d1-44e5-bc80-60e3f2fb378c admin admin] PATH_INFO: `/v3/projects/b06ad67dd0264b07b4e3319e7ede54b1/users/af170bbf9acf4519a22394dd5b1b69ae/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:22.417728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3d623-7d9c-4bef-9072-3840631edfd0 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:22.422674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbdea9-9e6f-41bc-9015-df6e35540306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:22.423567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbdea9-9e6f-41bc-9015-df6e35540306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:22.423849 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbdea9-9e6f-41bc-9015-df6e35540306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:22.424127 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbdea9-9e6f-41bc-9015-df6e35540306 admin admin] PATH_INFO: `/v3/users/3734082fdda041d281318bb7997da87d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:22.425428 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afbdea9-9e6f-41bc-9015-df6e35540306 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3734082fdda041d281318bb7997da87d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:22.427308 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afbdea9-9e6f-41bc-9015-df6e35540306 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:22.431210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6053b5e-76d1-44e5-bc80-60e3f2fb378c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b06ad67dd0264b07b4e3319e7ede54b1, user_id=af170bbf9acf4519a22394dd5b1b69ae, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:22.433952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6053b5e-76d1-44e5-bc80-60e3f2fb378c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:22.451524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:22.455091 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1012/2049] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:22 2022] PUT /identity/v3/projects/b06ad67dd0264b07b4e3319e7ede54b1/users/af170bbf9acf4519a22394dd5b1b69ae/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:22.465653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce77dc-ce3d-4314-97ef-a9596ff3145b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:22.465950 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce77dc-ce3d-4314-97ef-a9596ff3145b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:22.466412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce77dc-ce3d-4314-97ef-a9596ff3145b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:22.494100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7afbdea9-9e6f-41bc-9015-df6e35540306 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3734082fdda041d281318bb7997da87d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:22.495377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5bce77dc-ce3d-4314-97ef-a9596ff3145b None None] MFA Rules not processed for user `af170bbf9acf4519a22394dd5b1b69ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:22.496714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5bce77dc-ce3d-4314-97ef-a9596ff3145b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:22.500745 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7afbdea9-9e6f-41bc-9015-df6e35540306 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3734082fdda041d281318bb7997da87d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:22.501451 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7afbdea9-9e6f-41bc-9015-df6e35540306 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3734082fdda041d281318bb7997da87d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:22.510680 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7afbdea9-9e6f-41bc-9015-df6e35540306 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3734082fdda041d281318bb7997da87d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:22.511307 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7afbdea9-9e6f-41bc-9015-df6e35540306 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3734082fdda041d281318bb7997da87d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:22.534212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1038/2050] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:22 2022] DELETE /identity/v3/users/3734082fdda041d281318bb7997da87d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:22.538800 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47eb37d-7686-446a-8422-b3228289c211 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:22.539094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47eb37d-7686-446a-8422-b3228289c211 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:22.539384 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47eb37d-7686-446a-8422-b3228289c211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:22.558847 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a47eb37d-7686-446a-8422-b3228289c211 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:22.560072 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a47eb37d-7686-446a-8422-b3228289c211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:22.576874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1013/2051] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:22.581667 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce77dc-ce3d-4314-97ef-a9596ff3145b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:22.583332 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5bce77dc-ce3d-4314-97ef-a9596ff3145b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:22.595015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:22.646358 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1039/2052] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:07:22.655463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47eb37d-7686-446a-8422-b3228289c211 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:22.656829 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a47eb37d-7686-446a-8422-b3228289c211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:22.712580 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce77dc-ce3d-4314-97ef-a9596ff3145b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:22.721804 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a6834-eb53-4d25-a06b-5e8bb2d28c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:22.722684 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a6834-eb53-4d25-a06b-5e8bb2d28c1c admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:22.723256 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a6834-eb53-4d25-a06b-5e8bb2d28c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:22.723752 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a6834-eb53-4d25-a06b-5e8bb2d28c1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:22.724639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a6834-eb53-4d25-a06b-5e8bb2d28c1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:22.730770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47eb37d-7686-446a-8422-b3228289c211 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:22.732731 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a6834-eb53-4d25-a06b-5e8bb2d28c1c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:22.735716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36180b9-64a1-4bbd-95da-85947c7b88eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:22.736383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36180b9-64a1-4bbd-95da-85947c7b88eb service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:22.736730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36180b9-64a1-4bbd-95da-85947c7b88eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:22.737149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36180b9-64a1-4bbd-95da-85947c7b88eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:22.750842 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36180b9-64a1-4bbd-95da-85947c7b88eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:22.752299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36180b9-64a1-4bbd-95da-85947c7b88eb service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:22.779543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1040/2053] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:22.796581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36180b9-64a1-4bbd-95da-85947c7b88eb service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:22.796832 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1014/2054] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:22 2022] POST /identity/v3/projects => generated 385 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:07:22.807712 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36180b9-64a1-4bbd-95da-85947c7b88eb service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:22.918065 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:22.944581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5766cb0-3f82-420e-a1af-0d7a2f9a4e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:22.945694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5766cb0-3f82-420e-a1af-0d7a2f9a4e27 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:22.946156 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5766cb0-3f82-420e-a1af-0d7a2f9a4e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:22.946647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5766cb0-3f82-420e-a1af-0d7a2f9a4e27 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:22.947646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5766cb0-3f82-420e-a1af-0d7a2f9a4e27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:22.950313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5766cb0-3f82-420e-a1af-0d7a2f9a4e27 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:22.980478 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1041/2055] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:22 2022] POST /identity/v3/users => generated 299 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:07:22.981213 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a6834-eb53-4d25-a06b-5e8bb2d28c1c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:22.993342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a6834-eb53-4d25-a06b-5e8bb2d28c1c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:22.998852 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d00eaf-eebc-4385-9461-be3a1a25dbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:22.999642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d00eaf-eebc-4385-9461-be3a1a25dbc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:23.000076 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d00eaf-eebc-4385-9461-be3a1a25dbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:23.000743 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d00eaf-eebc-4385-9461-be3a1a25dbc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:23.001577 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d00eaf-eebc-4385-9461-be3a1a25dbc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:23.003284 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d00eaf-eebc-4385-9461-be3a1a25dbc3 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:23.014940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1015/2056] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:23.019620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5766cb0-3f82-420e-a1af-0d7a2f9a4e27 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:23.035006 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5766cb0-3f82-420e-a1af-0d7a2f9a4e27 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:23.040547 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5156c-2818-4f7b-8f38-48a673f2fe2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:23.041465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5156c-2818-4f7b-8f38-48a673f2fe2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:23.041750 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5156c-2818-4f7b-8f38-48a673f2fe2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:23.042100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5156c-2818-4f7b-8f38-48a673f2fe2e admin admin] PATH_INFO: `/v3/projects/ea0c3cb215dc418eb76f867a538c41d4/users/90414b65b2214da6832e509a976570bf/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:23.067248 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5156c-2818-4f7b-8f38-48a673f2fe2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea0c3cb215dc418eb76f867a538c41d4, user_id=90414b65b2214da6832e509a976570bf, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:23.070934 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5156c-2818-4f7b-8f38-48a673f2fe2e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:23.095657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1042/2057] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:23 2022] PUT /identity/v3/projects/ea0c3cb215dc418eb76f867a538c41d4/users/90414b65b2214da6832e509a976570bf/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:23.099759 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d00eaf-eebc-4385-9461-be3a1a25dbc3 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:23.111148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d00eaf-eebc-4385-9461-be3a1a25dbc3 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:23.197131 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5156c-2818-4f7b-8f38-48a673f2fe2e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:23.199002 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6af5156c-2818-4f7b-8f38-48a673f2fe2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:23.242015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91010526-e262-40b3-9a1f-40fc8b1f7e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:23.248699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91010526-e262-40b3-9a1f-40fc8b1f7e5c admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:23.248699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91010526-e262-40b3-9a1f-40fc8b1f7e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:23.248699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91010526-e262-40b3-9a1f-40fc8b1f7e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:23.249777 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91010526-e262-40b3-9a1f-40fc8b1f7e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:23.251307 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91010526-e262-40b3-9a1f-40fc8b1f7e5c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:23.269715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1016/2058] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:23.273589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91010526-e262-40b3-9a1f-40fc8b1f7e5c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:23.285710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91010526-e262-40b3-9a1f-40fc8b1f7e5c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:23.291089 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812e889-a2e5-405f-b6cc-6ce4379a3084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:23.292034 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812e889-a2e5-405f-b6cc-6ce4379a3084 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:23.292313 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812e889-a2e5-405f-b6cc-6ce4379a3084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:23.292626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812e889-a2e5-405f-b6cc-6ce4379a3084 admin admin] PATH_INFO: `/v3/projects/ea0c3cb215dc418eb76f867a538c41d4/users/90414b65b2214da6832e509a976570bf/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:23.295647 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812e889-a2e5-405f-b6cc-6ce4379a3084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea0c3cb215dc418eb76f867a538c41d4, user_id=90414b65b2214da6832e509a976570bf, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:23.298311 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812e889-a2e5-405f-b6cc-6ce4379a3084 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:23.311507 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1017/2059] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:23 2022] PUT /identity/v3/projects/ea0c3cb215dc418eb76f867a538c41d4/users/90414b65b2214da6832e509a976570bf/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:07:23.321759 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83c4f7-0a54-4f1e-bbeb-cc0abd286e35 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:23.322427 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83c4f7-0a54-4f1e-bbeb-cc0abd286e35 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:23.322503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5156c-2818-4f7b-8f38-48a673f2fe2e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:23.322578 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83c4f7-0a54-4f1e-bbeb-cc0abd286e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:23.353680 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ea83c4f7-0a54-4f1e-bbeb-cc0abd286e35 None None] MFA Rules not processed for user `90414b65b2214da6832e509a976570bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:23.355522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ea83c4f7-0a54-4f1e-bbeb-cc0abd286e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:23.392744 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8fe34-e898-46a8-b93c-90d57c9485e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:23.393511 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8fe34-e898-46a8-b93c-90d57c9485e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:23.394850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8fe34-e898-46a8-b93c-90d57c9485e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:23.394850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8fe34-e898-46a8-b93c-90d57c9485e3 admin admin] PATH_INFO: `/v3/projects/658ef933ba61420da8cd9a54c3ec7bc7` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:23.395252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8fe34-e898-46a8-b93c-90d57c9485e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=658ef933ba61420da8cd9a54c3ec7bc7)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:23.397945 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8fe34-e898-46a8-b93c-90d57c9485e3 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:23.423403 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:23.449068 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1018/2060] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:07:23.461994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83c4f7-0a54-4f1e-bbeb-cc0abd286e35 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:23.463342 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ea83c4f7-0a54-4f1e-bbeb-cc0abd286e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:23.467433 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-87c8fe34-e898-46a8-b93c-90d57c9485e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '658ef933ba61420da8cd9a54c3ec7bc7', 'request_id': 'req-87c8fe34-e898-46a8-b93c-90d57c9485e3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:23.472834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:23.474180 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-87c8fe34-e898-46a8-b93c-90d57c9485e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '658ef933ba61420da8cd9a54c3ec7bc7', 'request_id': 'req-87c8fe34-e898-46a8-b93c-90d57c9485e3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:23.483191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-87c8fe34-e898-46a8-b93c-90d57c9485e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '658ef933ba61420da8cd9a54c3ec7bc7', 'request_id': 'req-87c8fe34-e898-46a8-b93c-90d57c9485e3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:23.504607 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-87c8fe34-e898-46a8-b93c-90d57c9485e3 admin admin] The token cache is being invalidate because project 658ef933ba61420da8cd9a54c3ec7bc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:23.505154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-87c8fe34-e898-46a8-b93c-90d57c9485e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:23.511937 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-87c8fe34-e898-46a8-b93c-90d57c9485e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:23.511937 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1043/2061] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:23 2022] DELETE /identity/v3/projects/658ef933ba61420da8cd9a54c3ec7bc7 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:23.516045 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8fe34-e898-46a8-b93c-90d57c9485e3 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:23.516045 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-87c8fe34-e898-46a8-b93c-90d57c9485e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:23.536156 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83c4f7-0a54-4f1e-bbeb-cc0abd286e35 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:23.590421 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813eac63-3da0-454f-80c5-f7374649c276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:23.591218 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8fe34-e898-46a8-b93c-90d57c9485e3 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:23.591386 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813eac63-3da0-454f-80c5-f7374649c276 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:23.591701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813eac63-3da0-454f-80c5-f7374649c276 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:23.592054 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813eac63-3da0-454f-80c5-f7374649c276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:23.593932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-813eac63-3da0-454f-80c5-f7374649c276 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:23.596311 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a1547-60c1-4bb8-922d-c946b834231e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:23.597312 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a1547-60c1-4bb8-922d-c946b834231e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:23.597684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a1547-60c1-4bb8-922d-c946b834231e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:23.598026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a1547-60c1-4bb8-922d-c946b834231e admin admin] PATH_INFO: `/v3/users/bcf152e117da4f618dd4321b3691e7a7` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:23.599373 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a1547-60c1-4bb8-922d-c946b834231e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcf152e117da4f618dd4321b3691e7a7)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:23.601513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a1547-60c1-4bb8-922d-c946b834231e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:23.661595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e75a1547-60c1-4bb8-922d-c946b834231e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcf152e117da4f618dd4321b3691e7a7'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:23.669544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e75a1547-60c1-4bb8-922d-c946b834231e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcf152e117da4f618dd4321b3691e7a7'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:23.670857 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e75a1547-60c1-4bb8-922d-c946b834231e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcf152e117da4f618dd4321b3691e7a7'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:23.674594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813eac63-3da0-454f-80c5-f7374649c276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:23.676445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813eac63-3da0-454f-80c5-f7374649c276 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:23.679984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e75a1547-60c1-4bb8-922d-c946b834231e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcf152e117da4f618dd4321b3691e7a7'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:23.681122 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e75a1547-60c1-4bb8-922d-c946b834231e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcf152e117da4f618dd4321b3691e7a7'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:23.689510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1019/2062] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:07:23.705870 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1044/2063] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:23 2022] DELETE /identity/v3/users/bcf152e117da4f618dd4321b3691e7a7 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:07:23.763120 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813eac63-3da0-454f-80c5-f7374649c276 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:23.764858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-813eac63-3da0-454f-80c5-f7374649c276 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:23.780997 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813eac63-3da0-454f-80c5-f7374649c276 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:23.852227 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:23.860029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23496523-b146-477f-b8a1-2a7961675fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:23.860922 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23496523-b146-477f-b8a1-2a7961675fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:23.861212 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23496523-b146-477f-b8a1-2a7961675fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:23.861824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23496523-b146-477f-b8a1-2a7961675fb1 admin admin] PATH_INFO: `/v3/users/af170bbf9acf4519a22394dd5b1b69ae` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:23.863209 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23496523-b146-477f-b8a1-2a7961675fb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af170bbf9acf4519a22394dd5b1b69ae)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:23.866058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23496523-b146-477f-b8a1-2a7961675fb1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:23.934501 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-23496523-b146-477f-b8a1-2a7961675fb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af170bbf9acf4519a22394dd5b1b69ae'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:23.935242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-23496523-b146-477f-b8a1-2a7961675fb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af170bbf9acf4519a22394dd5b1b69ae'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:23.961806 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-23496523-b146-477f-b8a1-2a7961675fb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af170bbf9acf4519a22394dd5b1b69ae'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:23.985468 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-23496523-b146-477f-b8a1-2a7961675fb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af170bbf9acf4519a22394dd5b1b69ae'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:23.986562 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-23496523-b146-477f-b8a1-2a7961675fb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af170bbf9acf4519a22394dd5b1b69ae'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.032675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1020/2064] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:23 2022] DELETE /identity/v3/users/af170bbf9acf4519a22394dd5b1b69ae => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:07:24.033990 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854c13d-7bc9-428f-95b8-17c8bb0b3097 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:24.034448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854c13d-7bc9-428f-95b8-17c8bb0b3097 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:24.034605 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854c13d-7bc9-428f-95b8-17c8bb0b3097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:24.073047 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9854c13d-7bc9-428f-95b8-17c8bb0b3097 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:24.074337 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9854c13d-7bc9-428f-95b8-17c8bb0b3097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:24.167674 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23496523-b146-477f-b8a1-2a7961675fb1 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:24.169779 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-23496523-b146-477f-b8a1-2a7961675fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:24.230577 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1045/2065] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:07:24.242590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854c13d-7bc9-428f-95b8-17c8bb0b3097 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:24.244233 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9854c13d-7bc9-428f-95b8-17c8bb0b3097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:24.260128 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:24.278480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23496523-b146-477f-b8a1-2a7961675fb1 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:24.285401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c354b0-71e0-4679-9bbf-334a7a690eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:24.287112 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c354b0-71e0-4679-9bbf-334a7a690eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:24.288156 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c354b0-71e0-4679-9bbf-334a7a690eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:24.288667 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c354b0-71e0-4679-9bbf-334a7a690eba admin admin] PATH_INFO: `/v3/projects/f74f5b14b747445a8302ccae24e3afd5` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:24.291535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c354b0-71e0-4679-9bbf-334a7a690eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=f74f5b14b747445a8302ccae24e3afd5)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:24.294416 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c354b0-71e0-4679-9bbf-334a7a690eba admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:24.352594 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854c13d-7bc9-428f-95b8-17c8bb0b3097 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:24.358766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4f5bb-ab33-479b-abf3-60952c290b3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:24.359882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4f5bb-ab33-479b-abf3-60952c290b3d service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:24.360191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4f5bb-ab33-479b-abf3-60952c290b3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:24.360582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4f5bb-ab33-479b-abf3-60952c290b3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:24.384505 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4f5bb-ab33-479b-abf3-60952c290b3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:24.386024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4f5bb-ab33-479b-abf3-60952c290b3d service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:24.396270 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-06c354b0-71e0-4679-9bbf-334a7a690eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f74f5b14b747445a8302ccae24e3afd5', 'request_id': 'req-06c354b0-71e0-4679-9bbf-334a7a690eba'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.402550 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1046/2066] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:24.403423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4f5bb-ab33-479b-abf3-60952c290b3d service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:24.404690 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-06c354b0-71e0-4679-9bbf-334a7a690eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f74f5b14b747445a8302ccae24e3afd5', 'request_id': 'req-06c354b0-71e0-4679-9bbf-334a7a690eba'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.404842 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4ba4f5bb-ab33-479b-abf3-60952c290b3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:24.417021 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-06c354b0-71e0-4679-9bbf-334a7a690eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f74f5b14b747445a8302ccae24e3afd5', 'request_id': 'req-06c354b0-71e0-4679-9bbf-334a7a690eba'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.418764 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4f5bb-ab33-479b-abf3-60952c290b3d service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:24.446570 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-06c354b0-71e0-4679-9bbf-334a7a690eba admin admin] The token cache is being invalidate because project f74f5b14b747445a8302ccae24e3afd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:24.447549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-06c354b0-71e0-4679-9bbf-334a7a690eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.448641 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-06c354b0-71e0-4679-9bbf-334a7a690eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.451555 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1021/2067] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:24 2022] DELETE /identity/v3/projects/f74f5b14b747445a8302ccae24e3afd5 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:07:24.456233 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c354b0-71e0-4679-9bbf-334a7a690eba admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:24.460094 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-06c354b0-71e0-4679-9bbf-334a7a690eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:24.477801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:24.513276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac59252-5455-483e-8471-905a524c05a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:24.515286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac59252-5455-483e-8471-905a524c05a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:24.515815 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac59252-5455-483e-8471-905a524c05a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:24.516094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac59252-5455-483e-8471-905a524c05a5 admin admin] PATH_INFO: `/v3/users/55cbc34b1ff34eb9af038065b2724507` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:24.517529 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac59252-5455-483e-8471-905a524c05a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55cbc34b1ff34eb9af038065b2724507)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:24.520172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac59252-5455-483e-8471-905a524c05a5 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:24.571295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c354b0-71e0-4679-9bbf-334a7a690eba admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:24.588218 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fac59252-5455-483e-8471-905a524c05a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55cbc34b1ff34eb9af038065b2724507'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.596426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fac59252-5455-483e-8471-905a524c05a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55cbc34b1ff34eb9af038065b2724507'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.597331 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fac59252-5455-483e-8471-905a524c05a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55cbc34b1ff34eb9af038065b2724507'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.598587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50763c0b-677a-4893-912d-24531e7969fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:24.600433 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50763c0b-677a-4893-912d-24531e7969fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:24.600869 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50763c0b-677a-4893-912d-24531e7969fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:24.601283 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50763c0b-677a-4893-912d-24531e7969fb admin admin] PATH_INFO: `/v3/users/dc131acf310040e19fe59c36fd23f413` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:24.602731 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50763c0b-677a-4893-912d-24531e7969fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc131acf310040e19fe59c36fd23f413)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:24.605928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50763c0b-677a-4893-912d-24531e7969fb admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:24.610053 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fac59252-5455-483e-8471-905a524c05a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55cbc34b1ff34eb9af038065b2724507'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.614854 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fac59252-5455-483e-8471-905a524c05a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55cbc34b1ff34eb9af038065b2724507'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.634636 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1047/2068] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:24 2022] DELETE /identity/v3/users/55cbc34b1ff34eb9af038065b2724507 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:24.639352 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ec039-bd45-4d66-a04b-807f9ce9bfe4 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:24.639673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ec039-bd45-4d66-a04b-807f9ce9bfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:24.640616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ec039-bd45-4d66-a04b-807f9ce9bfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:24.662498 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-577ec039-bd45-4d66-a04b-807f9ce9bfe4 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:24.664668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-577ec039-bd45-4d66-a04b-807f9ce9bfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:24.683352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-50763c0b-677a-4893-912d-24531e7969fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc131acf310040e19fe59c36fd23f413'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.684113 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-50763c0b-677a-4893-912d-24531e7969fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc131acf310040e19fe59c36fd23f413'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.689639 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:24.694843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-50763c0b-677a-4893-912d-24531e7969fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc131acf310040e19fe59c36fd23f413'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.704938 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-50763c0b-677a-4893-912d-24531e7969fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc131acf310040e19fe59c36fd23f413'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.705703 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-50763c0b-677a-4893-912d-24531e7969fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc131acf310040e19fe59c36fd23f413'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.729215 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1022/2069] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:24 2022] DELETE /identity/v3/users/dc131acf310040e19fe59c36fd23f413 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:24.747169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1048/2070] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:24.754523 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50763c0b-677a-4893-912d-24531e7969fb admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:24.756296 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-50763c0b-677a-4893-912d-24531e7969fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:24.776653 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ec039-bd45-4d66-a04b-807f9ce9bfe4 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:24.778777 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-577ec039-bd45-4d66-a04b-807f9ce9bfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:24.794928 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ec039-bd45-4d66-a04b-807f9ce9bfe4 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:24.850495 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50763c0b-677a-4893-912d-24531e7969fb admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:24.855304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc4ac4-2cf7-4b9d-98d6-133c10201134 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:24.860456 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc4ac4-2cf7-4b9d-98d6-133c10201134 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:24.860456 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc4ac4-2cf7-4b9d-98d6-133c10201134 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:24.860456 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc4ac4-2cf7-4b9d-98d6-133c10201134 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:24.879597 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746328b4-c53b-4d3a-91a5-8f526105d508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:24.880684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746328b4-c53b-4d3a-91a5-8f526105d508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:24.881103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746328b4-c53b-4d3a-91a5-8f526105d508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:24.881444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746328b4-c53b-4d3a-91a5-8f526105d508 admin admin] PATH_INFO: `/v3/projects/b06ad67dd0264b07b4e3319e7ede54b1` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:24.882667 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746328b4-c53b-4d3a-91a5-8f526105d508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b06ad67dd0264b07b4e3319e7ede54b1)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:24.885013 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746328b4-c53b-4d3a-91a5-8f526105d508 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:24.928931 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:24.953172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-746328b4-c53b-4d3a-91a5-8f526105d508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b06ad67dd0264b07b4e3319e7ede54b1', 'request_id': 'req-746328b4-c53b-4d3a-91a5-8f526105d508'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.953996 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc4ac4-2cf7-4b9d-98d6-133c10201134 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:24.955535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc4ac4-2cf7-4b9d-98d6-133c10201134 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:24.958801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-746328b4-c53b-4d3a-91a5-8f526105d508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b06ad67dd0264b07b4e3319e7ede54b1', 'request_id': 'req-746328b4-c53b-4d3a-91a5-8f526105d508'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.970785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-746328b4-c53b-4d3a-91a5-8f526105d508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b06ad67dd0264b07b4e3319e7ede54b1', 'request_id': 'req-746328b4-c53b-4d3a-91a5-8f526105d508'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.971085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1023/2071] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:24.984065 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-746328b4-c53b-4d3a-91a5-8f526105d508 admin admin] The token cache is being invalidate because project b06ad67dd0264b07b4e3319e7ede54b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:24.984359 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-746328b4-c53b-4d3a-91a5-8f526105d508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.985014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-746328b4-c53b-4d3a-91a5-8f526105d508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:24.986616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1049/2072] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:24 2022] DELETE /identity/v3/projects/b06ad67dd0264b07b4e3319e7ede54b1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:07:24.990948 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc4ac4-2cf7-4b9d-98d6-133c10201134 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:25.001099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc4ac4-2cf7-4b9d-98d6-133c10201134 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:25.062919 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7a4bf-ff91-4631-9cce-05fe0ce61d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:25.063884 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7a4bf-ff91-4631-9cce-05fe0ce61d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:25.064135 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7a4bf-ff91-4631-9cce-05fe0ce61d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:25.064396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7a4bf-ff91-4631-9cce-05fe0ce61d3d admin admin] PATH_INFO: `/v3/users/90414b65b2214da6832e509a976570bf` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:25.065766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f7a4bf-ff91-4631-9cce-05fe0ce61d3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=90414b65b2214da6832e509a976570bf)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:25.067777 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f7a4bf-ff91-4631-9cce-05fe0ce61d3d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:25.077948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746328b4-c53b-4d3a-91a5-8f526105d508 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:25.079608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-746328b4-c53b-4d3a-91a5-8f526105d508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:25.093400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746328b4-c53b-4d3a-91a5-8f526105d508 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:25.103110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258256ec-47c0-4e99-839a-3d879baaf4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:25.118147 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258256ec-47c0-4e99-839a-3d879baaf4e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:25.118547 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258256ec-47c0-4e99-839a-3d879baaf4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:25.118779 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258256ec-47c0-4e99-839a-3d879baaf4e7 admin admin] PATH_INFO: `/v3/projects/a98f540a420f4561980508bd762ebd52` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:25.120738 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258256ec-47c0-4e99-839a-3d879baaf4e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a98f540a420f4561980508bd762ebd52)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:25.123331 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258256ec-47c0-4e99-839a-3d879baaf4e7 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:25.152430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-33f7a4bf-ff91-4631-9cce-05fe0ce61d3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90414b65b2214da6832e509a976570bf'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.153529 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-33f7a4bf-ff91-4631-9cce-05fe0ce61d3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90414b65b2214da6832e509a976570bf'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.164348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-33f7a4bf-ff91-4631-9cce-05fe0ce61d3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90414b65b2214da6832e509a976570bf'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.173312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-33f7a4bf-ff91-4631-9cce-05fe0ce61d3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90414b65b2214da6832e509a976570bf'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.174023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-33f7a4bf-ff91-4631-9cce-05fe0ce61d3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90414b65b2214da6832e509a976570bf'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.194106 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-258256ec-47c0-4e99-839a-3d879baaf4e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a98f540a420f4561980508bd762ebd52', 'request_id': 'req-258256ec-47c0-4e99-839a-3d879baaf4e7'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.196320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:25.200592 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-258256ec-47c0-4e99-839a-3d879baaf4e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a98f540a420f4561980508bd762ebd52', 'request_id': 'req-258256ec-47c0-4e99-839a-3d879baaf4e7'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.209458 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1024/2073] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:24 2022] DELETE /identity/v3/users/90414b65b2214da6832e509a976570bf => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:25.209834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-258256ec-47c0-4e99-839a-3d879baaf4e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a98f540a420f4561980508bd762ebd52', 'request_id': 'req-258256ec-47c0-4e99-839a-3d879baaf4e7'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.226866 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-258256ec-47c0-4e99-839a-3d879baaf4e7 admin admin] The token cache is being invalidate because project a98f540a420f4561980508bd762ebd52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:25.227155 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-258256ec-47c0-4e99-839a-3d879baaf4e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.227807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-258256ec-47c0-4e99-839a-3d879baaf4e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.229470 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1050/2074] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:25 2022] DELETE /identity/v3/projects/a98f540a420f4561980508bd762ebd52 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:25.235689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf24a54-e8eb-4222-8616-cedeffde3d90 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:25.236108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf24a54-e8eb-4222-8616-cedeffde3d90 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:25.236522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf24a54-e8eb-4222-8616-cedeffde3d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:25.254599 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2cf24a54-e8eb-4222-8616-cedeffde3d90 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:25.255898 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2cf24a54-e8eb-4222-8616-cedeffde3d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:25.314960 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:25.329866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1025/2075] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:25.338489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca93f48-90ca-495b-b6f5-221cded780ac None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:25.338782 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca93f48-90ca-495b-b6f5-221cded780ac None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:25.339063 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca93f48-90ca-495b-b6f5-221cded780ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:25.358981 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-0ca93f48-90ca-495b-b6f5-221cded780ac None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:25.360386 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0ca93f48-90ca-495b-b6f5-221cded780ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:25.369526 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1051/2076] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:25.373876 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf24a54-e8eb-4222-8616-cedeffde3d90 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:25.383585 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf24a54-e8eb-4222-8616-cedeffde3d90 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:25.388569 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c87c8-a63a-4e47-abbf-bfb42ae20dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:25.389409 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c87c8-a63a-4e47-abbf-bfb42ae20dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:25.389905 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c87c8-a63a-4e47-abbf-bfb42ae20dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:25.390507 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c87c8-a63a-4e47-abbf-bfb42ae20dd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:25.391254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c87c8-a63a-4e47-abbf-bfb42ae20dd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:07:25.391564 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c87c8-a63a-4e47-abbf-bfb42ae20dd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:25.393298 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c87c8-a63a-4e47-abbf-bfb42ae20dd5 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:25.410587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1026/2077] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:07:25.411704 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca93f48-90ca-495b-b6f5-221cded780ac None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:25.421555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca93f48-90ca-495b-b6f5-221cded780ac None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:25.427775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d7435-fd65-44f2-87da-faf6036fb546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:25.428576 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d7435-fd65-44f2-87da-faf6036fb546 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:25.428788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d7435-fd65-44f2-87da-faf6036fb546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:25.429049 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d7435-fd65-44f2-87da-faf6036fb546 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:25.429762 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d7435-fd65-44f2-87da-faf6036fb546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:25.431664 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d7435-fd65-44f2-87da-faf6036fb546 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:25.467523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1052/2078] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:25 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:25.471733 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c87c8-a63a-4e47-abbf-bfb42ae20dd5 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:25.483274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c87c8-a63a-4e47-abbf-bfb42ae20dd5 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:25.548961 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d7435-fd65-44f2-87da-faf6036fb546 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:25.552111 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-407d7435-fd65-44f2-87da-faf6036fb546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:25.579222 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4b405-9ced-4ed0-af17-5489f0981e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:25.581315 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4b405-9ced-4ed0-af17-5489f0981e35 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:25.581740 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4b405-9ced-4ed0-af17-5489f0981e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:25.582153 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4b405-9ced-4ed0-af17-5489f0981e35 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:25.583113 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4b405-9ced-4ed0-af17-5489f0981e35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:25.585425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4b405-9ced-4ed0-af17-5489f0981e35 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:25.617721 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1027/2079] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:25 2022] POST /identity/v3/users => generated 291 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:25.622082 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4b405-9ced-4ed0-af17-5489f0981e35 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:25.633620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4b405-9ced-4ed0-af17-5489f0981e35 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:25.634771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d7435-fd65-44f2-87da-faf6036fb546 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:25.639709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1b17e-043e-49d6-a310-97dcd031b064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:25.639863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66a008-8ad1-43b4-86e6-ba757e1904dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:25.640346 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1b17e-043e-49d6-a310-97dcd031b064 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:25.640675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1b17e-043e-49d6-a310-97dcd031b064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:25.640940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1b17e-043e-49d6-a310-97dcd031b064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:25.641205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66a008-8ad1-43b4-86e6-ba757e1904dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:25.641579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66a008-8ad1-43b4-86e6-ba757e1904dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:25.641893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e1b17e-043e-49d6-a310-97dcd031b064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:25.642132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66a008-8ad1-43b4-86e6-ba757e1904dc admin admin] PATH_INFO: `/v3/projects/4de61fda8d504511ae316afd9133ae37` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:25.643440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66a008-8ad1-43b4-86e6-ba757e1904dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4de61fda8d504511ae316afd9133ae37)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:25.644024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e1b17e-043e-49d6-a310-97dcd031b064 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:25.645821 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66a008-8ad1-43b4-86e6-ba757e1904dc admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:25.656449 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1028/2080] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:25.659140 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1b17e-043e-49d6-a310-97dcd031b064 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:25.676848 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1b17e-043e-49d6-a310-97dcd031b064 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:25.685983 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cad63-c113-4e49-ba40-53eab8c2d88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:25.686663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cad63-c113-4e49-ba40-53eab8c2d88b admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:25.686947 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cad63-c113-4e49-ba40-53eab8c2d88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:25.687264 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cad63-c113-4e49-ba40-53eab8c2d88b admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:25.688071 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291cad63-c113-4e49-ba40-53eab8c2d88b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:25.693063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291cad63-c113-4e49-ba40-53eab8c2d88b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:25.701746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3b66a008-8ad1-43b4-86e6-ba757e1904dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4de61fda8d504511ae316afd9133ae37', 'request_id': 'req-3b66a008-8ad1-43b4-86e6-ba757e1904dc'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.707954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3b66a008-8ad1-43b4-86e6-ba757e1904dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4de61fda8d504511ae316afd9133ae37', 'request_id': 'req-3b66a008-8ad1-43b4-86e6-ba757e1904dc'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.712719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1029/2081] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:25.713835 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cad63-c113-4e49-ba40-53eab8c2d88b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:25.715474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3b66a008-8ad1-43b4-86e6-ba757e1904dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4de61fda8d504511ae316afd9133ae37', 'request_id': 'req-3b66a008-8ad1-43b4-86e6-ba757e1904dc'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.717267 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-291cad63-c113-4e49-ba40-53eab8c2d88b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:25.735607 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cad63-c113-4e49-ba40-53eab8c2d88b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:25.738297 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3b66a008-8ad1-43b4-86e6-ba757e1904dc admin admin] The token cache is being invalidate because project 4de61fda8d504511ae316afd9133ae37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:25.738581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3b66a008-8ad1-43b4-86e6-ba757e1904dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.739322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3b66a008-8ad1-43b4-86e6-ba757e1904dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.741775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1053/2082] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:25 2022] DELETE /identity/v3/projects/4de61fda8d504511ae316afd9133ae37 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:07:25.745030 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66a008-8ad1-43b4-86e6-ba757e1904dc admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:25.745030 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3b66a008-8ad1-43b4-86e6-ba757e1904dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:25.794732 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:25.810302 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686114c8-6a11-48b0-83e2-8cd2c5dd7e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:25.811171 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686114c8-6a11-48b0-83e2-8cd2c5dd7e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:25.811456 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686114c8-6a11-48b0-83e2-8cd2c5dd7e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:25.812221 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686114c8-6a11-48b0-83e2-8cd2c5dd7e04 admin admin] PATH_INFO: `/v3/projects/ea0c3cb215dc418eb76f867a538c41d4` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:25.812955 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686114c8-6a11-48b0-83e2-8cd2c5dd7e04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea0c3cb215dc418eb76f867a538c41d4)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:25.814910 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686114c8-6a11-48b0-83e2-8cd2c5dd7e04 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:25.815519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66a008-8ad1-43b4-86e6-ba757e1904dc admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:25.820309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bce252-40a9-4efc-b5cd-437ebaeaa11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:25.821227 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bce252-40a9-4efc-b5cd-437ebaeaa11e admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:25.821506 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bce252-40a9-4efc-b5cd-437ebaeaa11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:25.821780 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bce252-40a9-4efc-b5cd-437ebaeaa11e admin admin] PATH_INFO: `/v3/projects/1e5e5870c28b4400ade619e753e67f7b/users/c5dc60b3eb544b428b43fe390b959d83/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:25.836772 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bce252-40a9-4efc-b5cd-437ebaeaa11e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e5e5870c28b4400ade619e753e67f7b, user_id=c5dc60b3eb544b428b43fe390b959d83, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:25.839418 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bce252-40a9-4efc-b5cd-437ebaeaa11e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:25.854272 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1054/2083] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:25 2022] PUT /identity/v3/projects/1e5e5870c28b4400ade619e753e67f7b/users/c5dc60b3eb544b428b43fe390b959d83/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:25.855521 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14be384-321d-4f99-bbd4-f9e0f2b51c49 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:25.855811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14be384-321d-4f99-bbd4-f9e0f2b51c49 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:25.856743 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14be384-321d-4f99-bbd4-f9e0f2b51c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:25.873020 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-686114c8-6a11-48b0-83e2-8cd2c5dd7e04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea0c3cb215dc418eb76f867a538c41d4', 'request_id': 'req-686114c8-6a11-48b0-83e2-8cd2c5dd7e04'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.875467 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:25.879863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-686114c8-6a11-48b0-83e2-8cd2c5dd7e04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea0c3cb215dc418eb76f867a538c41d4', 'request_id': 'req-686114c8-6a11-48b0-83e2-8cd2c5dd7e04'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.880514 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b14be384-321d-4f99-bbd4-f9e0f2b51c49 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:25.881919 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b14be384-321d-4f99-bbd4-f9e0f2b51c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:25.887825 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-686114c8-6a11-48b0-83e2-8cd2c5dd7e04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea0c3cb215dc418eb76f867a538c41d4', 'request_id': 'req-686114c8-6a11-48b0-83e2-8cd2c5dd7e04'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.901619 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-686114c8-6a11-48b0-83e2-8cd2c5dd7e04 admin admin] The token cache is being invalidate because project ea0c3cb215dc418eb76f867a538c41d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:25.901843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-686114c8-6a11-48b0-83e2-8cd2c5dd7e04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.902481 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-686114c8-6a11-48b0-83e2-8cd2c5dd7e04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:25.904070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1030/2084] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:25 2022] DELETE /identity/v3/projects/ea0c3cb215dc418eb76f867a538c41d4 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:25.905323 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad4fb7-d906-4adf-ae43-f20520c6c97a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:25.905605 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad4fb7-d906-4adf-ae43-f20520c6c97a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:25.905883 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad4fb7-d906-4adf-ae43-f20520c6c97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:25.933476 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-cfad4fb7-d906-4adf-ae43-f20520c6c97a None None] MFA Rules not processed for user `c5dc60b3eb544b428b43fe390b959d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:25.934822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-cfad4fb7-d906-4adf-ae43-f20520c6c97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:25.945696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1055/2085] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:25.947058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b7861-e114-40d3-ade2-9b1522055224 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:25.947346 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b7861-e114-40d3-ade2-9b1522055224 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:25.947621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b7861-e114-40d3-ade2-9b1522055224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:25.965445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9a4b7861-e114-40d3-ade2-9b1522055224 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:25.966651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9a4b7861-e114-40d3-ade2-9b1522055224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:25.975067 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1056/2086] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:25.976447 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70962c85-f157-431b-9736-416f005e65aa None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:25.976764 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70962c85-f157-431b-9736-416f005e65aa None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:25.976980 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70962c85-f157-431b-9736-416f005e65aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:25.994781 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-70962c85-f157-431b-9736-416f005e65aa None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:25.995981 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-70962c85-f157-431b-9736-416f005e65aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:26.003728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1057/2087] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:26.005153 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40682c51-d2d1-4db2-aac1-0122996d882c None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.005436 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40682c51-d2d1-4db2-aac1-0122996d882c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.005714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40682c51-d2d1-4db2-aac1-0122996d882c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.010164 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1031/2088] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:26.011030 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad4fb7-d906-4adf-ae43-f20520c6c97a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.023467 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad4fb7-d906-4adf-ae43-f20520c6c97a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.027348 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-40682c51-d2d1-4db2-aac1-0122996d882c None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:26.028578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-40682c51-d2d1-4db2-aac1-0122996d882c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:26.028690 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f51f0-ae81-450b-9af3-c897f12d6060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.029245 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f51f0-ae81-450b-9af3-c897f12d6060 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.029527 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f51f0-ae81-450b-9af3-c897f12d6060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.029801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f51f0-ae81-450b-9af3-c897f12d6060 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.030539 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f51f0-ae81-450b-9af3-c897f12d6060 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:07:26.030847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f51f0-ae81-450b-9af3-c897f12d6060 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:26.032499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f51f0-ae81-450b-9af3-c897f12d6060 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:26.036934 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1058/2089] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:26.037988 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40682c51-d2d1-4db2-aac1-0122996d882c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.039478 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-40682c51-d2d1-4db2-aac1-0122996d882c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:26.045030 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1032/2090] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:07:26.048808 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f51f0-ae81-450b-9af3-c897f12d6060 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.052831 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40682c51-d2d1-4db2-aac1-0122996d882c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.057934 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648a1a6-0f98-431e-9c11-6dacf7f3b4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.058324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f51f0-ae81-450b-9af3-c897f12d6060 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.058628 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648a1a6-0f98-431e-9c11-6dacf7f3b4ec admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.058923 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648a1a6-0f98-431e-9c11-6dacf7f3b4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.059203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648a1a6-0f98-431e-9c11-6dacf7f3b4ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.059947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f648a1a6-0f98-431e-9c11-6dacf7f3b4ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:26.061857 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f648a1a6-0f98-431e-9c11-6dacf7f3b4ec admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:26.063221 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c3a74-6f8b-4c71-8268-cfe180fdf9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.063865 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c3a74-6f8b-4c71-8268-cfe180fdf9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.064141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c3a74-6f8b-4c71-8268-cfe180fdf9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.064428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c3a74-6f8b-4c71-8268-cfe180fdf9ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.065176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508c3a74-6f8b-4c71-8268-cfe180fdf9ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:07:26.065476 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508c3a74-6f8b-4c71-8268-cfe180fdf9ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:26.067052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508c3a74-6f8b-4c71-8268-cfe180fdf9ee admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:26.079709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1033/2091] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:07:26.080681 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c3a74-6f8b-4c71-8268-cfe180fdf9ee admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.091103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1059/2092] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:26 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:26.091859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c3a74-6f8b-4c71-8268-cfe180fdf9ee admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.092546 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648a1a6-0f98-431e-9c11-6dacf7f3b4ec admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.149724 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22608bbc-068f-463c-b5da-bd49c4341c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.150396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22608bbc-068f-463c-b5da-bd49c4341c7e admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.150680 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22608bbc-068f-463c-b5da-bd49c4341c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.150954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22608bbc-068f-463c-b5da-bd49c4341c7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.151652 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22608bbc-068f-463c-b5da-bd49c4341c7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:26.153606 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22608bbc-068f-463c-b5da-bd49c4341c7e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:26.155523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648a1a6-0f98-431e-9c11-6dacf7f3b4ec admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.160767 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0da976-af90-450d-9582-7277f77ac67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.161766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0da976-af90-450d-9582-7277f77ac67f admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.162041 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0da976-af90-450d-9582-7277f77ac67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.162497 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0da976-af90-450d-9582-7277f77ac67f admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.163213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0da976-af90-450d-9582-7277f77ac67f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:26.165252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0da976-af90-450d-9582-7277f77ac67f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:26.183520 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1034/2093] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:26 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:26.183520 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22608bbc-068f-463c-b5da-bd49c4341c7e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.193932 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1060/2094] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:26 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:26.194942 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0da976-af90-450d-9582-7277f77ac67f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.208270 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:26.212553 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:26.246606 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22608bbc-068f-463c-b5da-bd49c4341c7e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.265519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0da976-af90-450d-9582-7277f77ac67f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.276505 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e282e-8ce2-4a52-a64f-233a86b6ac9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.277209 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e282e-8ce2-4a52-a64f-233a86b6ac9c admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.277478 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e282e-8ce2-4a52-a64f-233a86b6ac9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.277756 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e282e-8ce2-4a52-a64f-233a86b6ac9c admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.278454 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e282e-8ce2-4a52-a64f-233a86b6ac9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:26.280401 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e282e-8ce2-4a52-a64f-233a86b6ac9c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:26.305843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5454d-e6e1-472b-b51a-f0f95011b431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.306707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5454d-e6e1-472b-b51a-f0f95011b431 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.307007 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5454d-e6e1-472b-b51a-f0f95011b431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.307329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5454d-e6e1-472b-b51a-f0f95011b431 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.308049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5454d-e6e1-472b-b51a-f0f95011b431 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:26.310178 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1061/2095] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:26 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:26.310287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5454d-e6e1-472b-b51a-f0f95011b431 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:26.310884 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e282e-8ce2-4a52-a64f-233a86b6ac9c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.321129 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e282e-8ce2-4a52-a64f-233a86b6ac9c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.327552 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2514e83-a849-46f3-a6d8-384f4810bd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.328208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2514e83-a849-46f3-a6d8-384f4810bd9a admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.328517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2514e83-a849-46f3-a6d8-384f4810bd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.328821 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2514e83-a849-46f3-a6d8-384f4810bd9a admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.329470 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2514e83-a849-46f3-a6d8-384f4810bd9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:26.331376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2514e83-a849-46f3-a6d8-384f4810bd9a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:26.332483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1035/2096] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:26 2022] POST /identity/v3/users => generated 298 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:26.333286 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5454d-e6e1-472b-b51a-f0f95011b431 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.344178 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5454d-e6e1-472b-b51a-f0f95011b431 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.349265 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f7254-0cd1-4efe-a12f-9d3603a72278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.349953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f7254-0cd1-4efe-a12f-9d3603a72278 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.350230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f7254-0cd1-4efe-a12f-9d3603a72278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.350499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f7254-0cd1-4efe-a12f-9d3603a72278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.351226 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f7254-0cd1-4efe-a12f-9d3603a72278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:26.354705 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1062/2097] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:26 2022] POST /identity/v3/users => generated 294 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:26.354705 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2514e83-a849-46f3-a6d8-384f4810bd9a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.354950 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f7254-0cd1-4efe-a12f-9d3603a72278 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:26.363776 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1036/2098] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:26.364672 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f7254-0cd1-4efe-a12f-9d3603a72278 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.364780 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2514e83-a849-46f3-a6d8-384f4810bd9a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.369649 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22a450-690d-44b2-ac89-697210c9bf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.370306 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22a450-690d-44b2-ac89-697210c9bf1b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.370583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22a450-690d-44b2-ac89-697210c9bf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.370858 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22a450-690d-44b2-ac89-697210c9bf1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.371580 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d22a450-690d-44b2-ac89-697210c9bf1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:26.373376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d22a450-690d-44b2-ac89-697210c9bf1b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:26.374952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f7254-0cd1-4efe-a12f-9d3603a72278 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.379579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f467a4a-c3e1-4c4b-9734-c6ee0c1a7555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.380225 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f467a4a-c3e1-4c4b-9734-c6ee0c1a7555 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.380544 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f467a4a-c3e1-4c4b-9734-c6ee0c1a7555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.380962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f467a4a-c3e1-4c4b-9734-c6ee0c1a7555 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.381546 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f467a4a-c3e1-4c4b-9734-c6ee0c1a7555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:26.383284 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f467a4a-c3e1-4c4b-9734-c6ee0c1a7555 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:26.387364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1063/2099] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:26.387364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22a450-690d-44b2-ac89-697210c9bf1b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.394659 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1037/2100] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:26.394782 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22a450-690d-44b2-ac89-697210c9bf1b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.395426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f467a4a-c3e1-4c4b-9734-c6ee0c1a7555 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.401655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f96b7-a680-489f-96c4-da75fd53eb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.402304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f96b7-a680-489f-96c4-da75fd53eb94 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.402580 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f96b7-a680-489f-96c4-da75fd53eb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.402851 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f96b7-a680-489f-96c4-da75fd53eb94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.403567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f96b7-a680-489f-96c4-da75fd53eb94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:26.405175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f96b7-a680-489f-96c4-da75fd53eb94 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:26.405591 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f467a4a-c3e1-4c4b-9734-c6ee0c1a7555 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.410741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d696f61-3141-4b3c-abde-f33f7656b126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.411325 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d696f61-3141-4b3c-abde-f33f7656b126 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.411608 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d696f61-3141-4b3c-abde-f33f7656b126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.411876 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d696f61-3141-4b3c-abde-f33f7656b126 admin admin] PATH_INFO: `/v3/projects/74f834722df641f0ba6a44f86d4e1db4/users/2eee7044df9b45d6bc32f716d1df5c83/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.419562 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1064/2101] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:26.419562 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f96b7-a680-489f-96c4-da75fd53eb94 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.427217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f96b7-a680-489f-96c4-da75fd53eb94 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.427465 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d696f61-3141-4b3c-abde-f33f7656b126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74f834722df641f0ba6a44f86d4e1db4, user_id=2eee7044df9b45d6bc32f716d1df5c83, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:26.430597 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d696f61-3141-4b3c-abde-f33f7656b126 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:26.432245 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c9cee-491d-4f4e-9a68-682fa6e4c1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.432934 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c9cee-491d-4f4e-9a68-682fa6e4c1e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.433209 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c9cee-491d-4f4e-9a68-682fa6e4c1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.433819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c9cee-491d-4f4e-9a68-682fa6e4c1e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.434189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c9cee-491d-4f4e-9a68-682fa6e4c1e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:26.436067 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c9cee-491d-4f4e-9a68-682fa6e4c1e3 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:26.444734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1038/2102] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:26 2022] PUT /identity/v3/projects/74f834722df641f0ba6a44f86d4e1db4/users/2eee7044df9b45d6bc32f716d1df5c83/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:26.446421 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d696f61-3141-4b3c-abde-f33f7656b126 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.452648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1065/2103] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:26.452648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c9cee-491d-4f4e-9a68-682fa6e4c1e3 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.460054 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c9cee-491d-4f4e-9a68-682fa6e4c1e3 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.514364 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d696f61-3141-4b3c-abde-f33f7656b126 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.517504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6a3d0-b4fa-41f9-85ae-debe5b52475e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.518187 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6a3d0-b4fa-41f9-85ae-debe5b52475e admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.518468 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6a3d0-b4fa-41f9-85ae-debe5b52475e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.518737 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6a3d0-b4fa-41f9-85ae-debe5b52475e admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.519455 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6a3d0-b4fa-41f9-85ae-debe5b52475e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:26.519565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b5eb5-6093-4b27-9fe6-0fb0900a455c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.520348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b5eb5-6093-4b27-9fe6-0fb0900a455c admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.520660 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b5eb5-6093-4b27-9fe6-0fb0900a455c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.520936 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b5eb5-6093-4b27-9fe6-0fb0900a455c admin admin] PATH_INFO: `/v3/projects/a60af8d79166435d9147690a742b5709/users/a8c449e64d80404aa10a19a645da2cc1/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.521075 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6a3d0-b4fa-41f9-85ae-debe5b52475e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:26.531762 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1066/2104] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:26.533561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6a3d0-b4fa-41f9-85ae-debe5b52475e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.541111 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b5eb5-6093-4b27-9fe6-0fb0900a455c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a60af8d79166435d9147690a742b5709, user_id=a8c449e64d80404aa10a19a645da2cc1, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:26.545053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b5eb5-6093-4b27-9fe6-0fb0900a455c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:26.548818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6a3d0-b4fa-41f9-85ae-debe5b52475e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.555393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538e6e6-3a57-47ac-bffc-b27ad5691f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.556417 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538e6e6-3a57-47ac-bffc-b27ad5691f59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.556877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538e6e6-3a57-47ac-bffc-b27ad5691f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.557159 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538e6e6-3a57-47ac-bffc-b27ad5691f59 admin admin] PATH_INFO: `/v3/projects/2a2c933d1a734104941a0db6e46d13e5/users/d5c173d6b4a24861b60a91cf358ac84d/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.565297 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1039/2105] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:26 2022] PUT /identity/v3/projects/a60af8d79166435d9147690a742b5709/users/a8c449e64d80404aa10a19a645da2cc1/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:26.567343 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b5eb5-6093-4b27-9fe6-0fb0900a455c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.574326 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538e6e6-3a57-47ac-bffc-b27ad5691f59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a2c933d1a734104941a0db6e46d13e5, user_id=d5c173d6b4a24861b60a91cf358ac84d, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:26.577200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538e6e6-3a57-47ac-bffc-b27ad5691f59 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:26.580052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:26.593105 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1067/2106] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:26 2022] PUT /identity/v3/projects/2a2c933d1a734104941a0db6e46d13e5/users/d5c173d6b4a24861b60a91cf358ac84d/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:26.595087 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66c254-02d1-4042-bed7-26ed80ed589b None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.595384 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66c254-02d1-4042-bed7-26ed80ed589b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.595666 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66c254-02d1-4042-bed7-26ed80ed589b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.627331 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-fd66c254-02d1-4042-bed7-26ed80ed589b None None] MFA Rules not processed for user `a8c449e64d80404aa10a19a645da2cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:26.628575 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-fd66c254-02d1-4042-bed7-26ed80ed589b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:26.637302 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:26.652017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b5eb5-6093-4b27-9fe6-0fb0900a455c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.708337 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1068/2107] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:26.711959 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7ae23-34e3-4d6b-bf57-88a7b7578242 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.711959 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7ae23-34e3-4d6b-bf57-88a7b7578242 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.711959 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7ae23-34e3-4d6b-bf57-88a7b7578242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.721895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebf711-39ae-40d7-ad18-c4b23b5c2272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.722931 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebf711-39ae-40d7-ad18-c4b23b5c2272 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.723209 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebf711-39ae-40d7-ad18-c4b23b5c2272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.723801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebf711-39ae-40d7-ad18-c4b23b5c2272 admin admin] PATH_INFO: `/v3/projects/74f834722df641f0ba6a44f86d4e1db4/users/2eee7044df9b45d6bc32f716d1df5c83/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.726049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebf711-39ae-40d7-ad18-c4b23b5c2272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74f834722df641f0ba6a44f86d4e1db4, user_id=2eee7044df9b45d6bc32f716d1df5c83, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:26.729345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebf711-39ae-40d7-ad18-c4b23b5c2272 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:26.741658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1040/2108] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:26 2022] PUT /identity/v3/projects/74f834722df641f0ba6a44f86d4e1db4/users/2eee7044df9b45d6bc32f716d1df5c83/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:26.742641 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebf711-39ae-40d7-ad18-c4b23b5c2272 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.744457 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6cebf711-39ae-40d7-ad18-c4b23b5c2272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:26.752841 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1ce7ae23-34e3-4d6b-bf57-88a7b7578242 None None] MFA Rules not processed for user `d5c173d6b4a24861b60a91cf358ac84d`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:26.754054 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1ce7ae23-34e3-4d6b-bf57-88a7b7578242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:26.816731 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebf711-39ae-40d7-ad18-c4b23b5c2272 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.823141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ead7f-4f79-4aa4-a797-9ed33e7cb7b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.825153 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ead7f-4f79-4aa4-a797-9ed33e7cb7b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.825438 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ead7f-4f79-4aa4-a797-9ed33e7cb7b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.825714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ead7f-4f79-4aa4-a797-9ed33e7cb7b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.826190 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1069/2109] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:26.827476 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0fe80-112c-4bf1-98c3-3b35fda302df None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.827766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0fe80-112c-4bf1-98c3-3b35fda302df None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.828028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0fe80-112c-4bf1-98c3-3b35fda302df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.857674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-dba0fe80-112c-4bf1-98c3-3b35fda302df None None] MFA Rules not processed for user `2eee7044df9b45d6bc32f716d1df5c83`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:26.858873 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-dba0fe80-112c-4bf1-98c3-3b35fda302df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:26.932805 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ead7f-4f79-4aa4-a797-9ed33e7cb7b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:26.934372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ead7f-4f79-4aa4-a797-9ed33e7cb7b0 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:26.934960 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1070/2110] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:26.937301 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0fe80-112c-4bf1-98c3-3b35fda302df None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.938513 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:26.946163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1041/2111] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:26.947179 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ead7f-4f79-4aa4-a797-9ed33e7cb7b0 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:26.949116 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e99ead7f-4f79-4aa4-a797-9ed33e7cb7b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:26.960332 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ead7f-4f79-4aa4-a797-9ed33e7cb7b0 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:26.964152 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5c8f0-60dc-4d74-bd2a-7ff2dbc311dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:26.964829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5c8f0-60dc-4d74-bd2a-7ff2dbc311dc service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:26.965104 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5c8f0-60dc-4d74-bd2a-7ff2dbc311dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:26.965370 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5c8f0-60dc-4d74-bd2a-7ff2dbc311dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:26.969183 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:27.009572 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0fe80-112c-4bf1-98c3-3b35fda302df None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:27.014898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade300da-0048-4517-ac8e-ca1f0b1d4afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:27.016123 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade300da-0048-4517-ac8e-ca1f0b1d4afc admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:27.016634 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade300da-0048-4517-ac8e-ca1f0b1d4afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:27.016978 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade300da-0048-4517-ac8e-ca1f0b1d4afc admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:27.017879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade300da-0048-4517-ac8e-ca1f0b1d4afc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:27.019835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade300da-0048-4517-ac8e-ca1f0b1d4afc admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:27.031966 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e5c8f0-60dc-4d74-bd2a-7ff2dbc311dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:27.033686 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e5c8f0-60dc-4d74-bd2a-7ff2dbc311dc service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:27.045151 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1042/2112] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:27.048714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1071/2113] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:26 2022] POST /identity/v3/projects => generated 389 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:27.053887 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5c8f0-60dc-4d74-bd2a-7ff2dbc311dc service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:27.067251 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5c8f0-60dc-4d74-bd2a-7ff2dbc311dc service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:27.131372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec4857-ea2c-4b4e-9bc9-f4b50ed7c80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:27.132082 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec4857-ea2c-4b4e-9bc9-f4b50ed7c80e admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:27.132357 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec4857-ea2c-4b4e-9bc9-f4b50ed7c80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:27.132660 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec4857-ea2c-4b4e-9bc9-f4b50ed7c80e admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:27.133357 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec4857-ea2c-4b4e-9bc9-f4b50ed7c80e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:27.135279 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec4857-ea2c-4b4e-9bc9-f4b50ed7c80e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:27.164114 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1043/2114] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:27 2022] POST /identity/v3/users => generated 301 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:27.166632 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade300da-0048-4517-ac8e-ca1f0b1d4afc admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:27.178281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade300da-0048-4517-ac8e-ca1f0b1d4afc admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:27.183902 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2bee9b-1d75-437d-98ce-9f9ba3dc3477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:27.184617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bee9b-1d75-437d-98ce-9f9ba3dc3477 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:27.184903 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bee9b-1d75-437d-98ce-9f9ba3dc3477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:27.185178 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bee9b-1d75-437d-98ce-9f9ba3dc3477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:27.185911 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2bee9b-1d75-437d-98ce-9f9ba3dc3477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:27.187515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2bee9b-1d75-437d-98ce-9f9ba3dc3477 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:27.199774 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1072/2115] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:07:27.204450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec4857-ea2c-4b4e-9bc9-f4b50ed7c80e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:27.219150 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec4857-ea2c-4b4e-9bc9-f4b50ed7c80e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:27.224006 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1416a-d9ec-46ec-8e03-7a1bd1c78f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:27.224970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1416a-d9ec-46ec-8e03-7a1bd1c78f9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:27.225560 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1416a-d9ec-46ec-8e03-7a1bd1c78f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:27.225852 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1416a-d9ec-46ec-8e03-7a1bd1c78f9b admin admin] PATH_INFO: `/v3/projects/3d727fa5bde046ea8bfee66876fbbb3d/users/a908b76985e44700a4beaa78754bb0fa/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:27.262008 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1416a-d9ec-46ec-8e03-7a1bd1c78f9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d727fa5bde046ea8bfee66876fbbb3d, user_id=a908b76985e44700a4beaa78754bb0fa, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:27.264746 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1416a-d9ec-46ec-8e03-7a1bd1c78f9b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:27.283993 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1044/2116] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:27 2022] PUT /identity/v3/projects/3d727fa5bde046ea8bfee66876fbbb3d/users/a908b76985e44700a4beaa78754bb0fa/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:27.290336 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2bee9b-1d75-437d-98ce-9f9ba3dc3477 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:27.304378 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2bee9b-1d75-437d-98ce-9f9ba3dc3477 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:27.411566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101fd15-c06a-4181-bed7-b241fe841e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:27.412763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101fd15-c06a-4181-bed7-b241fe841e2a admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:27.413049 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101fd15-c06a-4181-bed7-b241fe841e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:27.413581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101fd15-c06a-4181-bed7-b241fe841e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:27.414082 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101fd15-c06a-4181-bed7-b241fe841e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:27.415733 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101fd15-c06a-4181-bed7-b241fe841e2a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:27.429623 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1073/2117] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:07:27.434224 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1416a-d9ec-46ec-8e03-7a1bd1c78f9b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:27.445159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1416a-d9ec-46ec-8e03-7a1bd1c78f9b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:27.449978 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c2cb7-c492-4d9d-a660-b024eddd1f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:27.450877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c2cb7-c492-4d9d-a660-b024eddd1f74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:27.451160 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c2cb7-c492-4d9d-a660-b024eddd1f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:27.451438 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c2cb7-c492-4d9d-a660-b024eddd1f74 admin admin] PATH_INFO: `/v3/projects/3d727fa5bde046ea8bfee66876fbbb3d/users/a908b76985e44700a4beaa78754bb0fa/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:27.453567 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c2cb7-c492-4d9d-a660-b024eddd1f74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d727fa5bde046ea8bfee66876fbbb3d, user_id=a908b76985e44700a4beaa78754bb0fa, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:27.456712 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c2cb7-c492-4d9d-a660-b024eddd1f74 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:27.469792 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1045/2118] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:27 2022] PUT /identity/v3/projects/3d727fa5bde046ea8bfee66876fbbb3d/users/a908b76985e44700a4beaa78754bb0fa/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:07:27.480629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985fa31-da5f-4898-82b9-3082bdeee798 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:27.481033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985fa31-da5f-4898-82b9-3082bdeee798 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:27.481321 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985fa31-da5f-4898-82b9-3082bdeee798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:27.532925 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c985fa31-da5f-4898-82b9-3082bdeee798 None None] MFA Rules not processed for user `a908b76985e44700a4beaa78754bb0fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:27.534372 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c985fa31-da5f-4898-82b9-3082bdeee798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:27.618716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1074/2119] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:27.630396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c2cb7-c492-4d9d-a660-b024eddd1f74 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:27.686741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:27.692226 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c2cb7-c492-4d9d-a660-b024eddd1f74 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:27.700727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8f67e-7c39-4862-ae22-55a0097064d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:27.701387 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8f67e-7c39-4862-ae22-55a0097064d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:27.701668 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8f67e-7c39-4862-ae22-55a0097064d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:27.701926 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8f67e-7c39-4862-ae22-55a0097064d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:27.712197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8f67e-7c39-4862-ae22-55a0097064d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:27.713695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8f67e-7c39-4862-ae22-55a0097064d0 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:27.725428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1046/2120] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:27 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:07:27.867795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985fa31-da5f-4898-82b9-3082bdeee798 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:27.893754 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:27.947092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985fa31-da5f-4898-82b9-3082bdeee798 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:27.952655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401ba1c-1daf-4c13-ac73-4b08cd4ba9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:27.953569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401ba1c-1daf-4c13-ac73-4b08cd4ba9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:27.953850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401ba1c-1daf-4c13-ac73-4b08cd4ba9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:27.954150 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401ba1c-1daf-4c13-ac73-4b08cd4ba9b3 admin admin] PATH_INFO: `/v3/users/d5c173d6b4a24861b60a91cf358ac84d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:27.955440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401ba1c-1daf-4c13-ac73-4b08cd4ba9b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5c173d6b4a24861b60a91cf358ac84d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:27.957437 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401ba1c-1daf-4c13-ac73-4b08cd4ba9b3 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:28.012550 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c401ba1c-1daf-4c13-ac73-4b08cd4ba9b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5c173d6b4a24861b60a91cf358ac84d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:28.019819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c401ba1c-1daf-4c13-ac73-4b08cd4ba9b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5c173d6b4a24861b60a91cf358ac84d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:28.020674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c401ba1c-1daf-4c13-ac73-4b08cd4ba9b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5c173d6b4a24861b60a91cf358ac84d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:28.033087 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c401ba1c-1daf-4c13-ac73-4b08cd4ba9b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5c173d6b4a24861b60a91cf358ac84d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:28.033876 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c401ba1c-1daf-4c13-ac73-4b08cd4ba9b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5c173d6b4a24861b60a91cf358ac84d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:28.053850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1075/2121] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:27 2022] DELETE /identity/v3/users/d5c173d6b4a24861b60a91cf358ac84d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:28.058614 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd60d78-d12f-40e7-be81-e09b5c318eee None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:28.058987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd60d78-d12f-40e7-be81-e09b5c318eee None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:28.059328 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd60d78-d12f-40e7-be81-e09b5c318eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:28.077867 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-3cd60d78-d12f-40e7-be81-e09b5c318eee None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:28.079011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3cd60d78-d12f-40e7-be81-e09b5c318eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:28.146709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1047/2122] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:28.157601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401ba1c-1daf-4c13-ac73-4b08cd4ba9b3 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:28.159211 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c401ba1c-1daf-4c13-ac73-4b08cd4ba9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:28.220815 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401ba1c-1daf-4c13-ac73-4b08cd4ba9b3 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:28.225902 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a6c6f-a444-4a67-83e9-50d709da535b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:28.226798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a6c6f-a444-4a67-83e9-50d709da535b service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:28.227410 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a6c6f-a444-4a67-83e9-50d709da535b service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:28.227848 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a6c6f-a444-4a67-83e9-50d709da535b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:28.237874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a6c6f-a444-4a67-83e9-50d709da535b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:28.239312 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a6c6f-a444-4a67-83e9-50d709da535b service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:28.252160 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1076/2123] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:28.565433 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd60d78-d12f-40e7-be81-e09b5c318eee None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:28.566965 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3cd60d78-d12f-40e7-be81-e09b5c318eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:28.579308 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd60d78-d12f-40e7-be81-e09b5c318eee None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:28.591488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07baa275-8b17-4846-997e-e28b0d9b1e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:28.592372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baa275-8b17-4846-997e-e28b0d9b1e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:28.592683 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baa275-8b17-4846-997e-e28b0d9b1e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:28.592959 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baa275-8b17-4846-997e-e28b0d9b1e96 admin admin] PATH_INFO: `/v3/projects/2a2c933d1a734104941a0db6e46d13e5` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:28.594283 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07baa275-8b17-4846-997e-e28b0d9b1e96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a2c933d1a734104941a0db6e46d13e5)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:28.596204 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07baa275-8b17-4846-997e-e28b0d9b1e96 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:28.664827 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-07baa275-8b17-4846-997e-e28b0d9b1e96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a2c933d1a734104941a0db6e46d13e5', 'request_id': 'req-07baa275-8b17-4846-997e-e28b0d9b1e96'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:28.672838 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-07baa275-8b17-4846-997e-e28b0d9b1e96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a2c933d1a734104941a0db6e46d13e5', 'request_id': 'req-07baa275-8b17-4846-997e-e28b0d9b1e96'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:28.680263 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-07baa275-8b17-4846-997e-e28b0d9b1e96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a2c933d1a734104941a0db6e46d13e5', 'request_id': 'req-07baa275-8b17-4846-997e-e28b0d9b1e96'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:28.698707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-07baa275-8b17-4846-997e-e28b0d9b1e96 admin admin] The token cache is being invalidate because project 2a2c933d1a734104941a0db6e46d13e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:28.699092 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-07baa275-8b17-4846-997e-e28b0d9b1e96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:28.699727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-07baa275-8b17-4846-997e-e28b0d9b1e96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:28.701505 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1048/2124] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:28 2022] DELETE /identity/v3/projects/2a2c933d1a734104941a0db6e46d13e5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:28.705656 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a6c6f-a444-4a67-83e9-50d709da535b service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:28.715647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a6c6f-a444-4a67-83e9-50d709da535b service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:28.762141 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:28.803124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5670aece-9f21-4fe6-93d7-1a6b17631c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:28.804344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670aece-9f21-4fe6-93d7-1a6b17631c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:28.804756 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670aece-9f21-4fe6-93d7-1a6b17631c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:28.805678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670aece-9f21-4fe6-93d7-1a6b17631c42 admin admin] PATH_INFO: `/v3/users/a908b76985e44700a4beaa78754bb0fa` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:28.807398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5670aece-9f21-4fe6-93d7-1a6b17631c42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a908b76985e44700a4beaa78754bb0fa)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:28.810182 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5670aece-9f21-4fe6-93d7-1a6b17631c42 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:28.899762 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5670aece-9f21-4fe6-93d7-1a6b17631c42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a908b76985e44700a4beaa78754bb0fa'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:28.910276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5670aece-9f21-4fe6-93d7-1a6b17631c42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a908b76985e44700a4beaa78754bb0fa'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:28.911130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5670aece-9f21-4fe6-93d7-1a6b17631c42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a908b76985e44700a4beaa78754bb0fa'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:28.925210 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5670aece-9f21-4fe6-93d7-1a6b17631c42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a908b76985e44700a4beaa78754bb0fa'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:28.926384 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-5670aece-9f21-4fe6-93d7-1a6b17631c42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a908b76985e44700a4beaa78754bb0fa'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:28.950697 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1077/2125] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:28 2022] DELETE /identity/v3/users/a908b76985e44700a4beaa78754bb0fa => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:07:29.282996 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07baa275-8b17-4846-997e-e28b0d9b1e96 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:29.284567 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-07baa275-8b17-4846-997e-e28b0d9b1e96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:29.315046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07baa275-8b17-4846-997e-e28b0d9b1e96 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:29.385760 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e9915-62cf-4453-a473-3347c221cea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:29.386605 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e9915-62cf-4453-a473-3347c221cea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:29.386891 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e9915-62cf-4453-a473-3347c221cea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:29.387164 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e9915-62cf-4453-a473-3347c221cea9 admin admin] PATH_INFO: `/v3/projects/3d727fa5bde046ea8bfee66876fbbb3d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:29.388732 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e9915-62cf-4453-a473-3347c221cea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d727fa5bde046ea8bfee66876fbbb3d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:29.393098 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e9915-62cf-4453-a473-3347c221cea9 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:29.414382 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:29.450201 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-779e9915-62cf-4453-a473-3347c221cea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d727fa5bde046ea8bfee66876fbbb3d', 'request_id': 'req-779e9915-62cf-4453-a473-3347c221cea9'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:29.457016 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-779e9915-62cf-4453-a473-3347c221cea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d727fa5bde046ea8bfee66876fbbb3d', 'request_id': 'req-779e9915-62cf-4453-a473-3347c221cea9'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:29.465230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-779e9915-62cf-4453-a473-3347c221cea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d727fa5bde046ea8bfee66876fbbb3d', 'request_id': 'req-779e9915-62cf-4453-a473-3347c221cea9'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:29.485699 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-779e9915-62cf-4453-a473-3347c221cea9 admin admin] The token cache is being invalidate because project 3d727fa5bde046ea8bfee66876fbbb3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:29.486303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-779e9915-62cf-4453-a473-3347c221cea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:29.486994 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-779e9915-62cf-4453-a473-3347c221cea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:29.488856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1049/2126] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:29 2022] DELETE /identity/v3/projects/3d727fa5bde046ea8bfee66876fbbb3d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:29.494162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c52380-56af-4a3d-a1ed-19c60a1b5b6c None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:29.494657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c52380-56af-4a3d-a1ed-19c60a1b5b6c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:29.494657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c52380-56af-4a3d-a1ed-19c60a1b5b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:29.513893 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-23c52380-56af-4a3d-a1ed-19c60a1b5b6c None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:29.515438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-23c52380-56af-4a3d-a1ed-19c60a1b5b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:29.577787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1078/2127] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:29.586142 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46793a-6640-4b88-90c2-e90a472e2c66 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:29.586791 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46793a-6640-4b88-90c2-e90a472e2c66 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:29.586791 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46793a-6640-4b88-90c2-e90a472e2c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:29.609576 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-cb46793a-6640-4b88-90c2-e90a472e2c66 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:29.610804 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-cb46793a-6640-4b88-90c2-e90a472e2c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:29.620161 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1050/2128] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:29.624656 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c52380-56af-4a3d-a1ed-19c60a1b5b6c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:29.634593 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c52380-56af-4a3d-a1ed-19c60a1b5b6c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:29.639266 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4bb544-ab62-4e31-a5bd-13613320b701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:29.639931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bb544-ab62-4e31-a5bd-13613320b701 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:29.640569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bb544-ab62-4e31-a5bd-13613320b701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:29.640848 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4bb544-ab62-4e31-a5bd-13613320b701 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:29.641582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4bb544-ab62-4e31-a5bd-13613320b701 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:07:29.641894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4bb544-ab62-4e31-a5bd-13613320b701 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:29.643523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4bb544-ab62-4e31-a5bd-13613320b701 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:29.655701 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1079/2129] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:07:29.660023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46793a-6640-4b88-90c2-e90a472e2c66 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:29.670111 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46793a-6640-4b88-90c2-e90a472e2c66 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:29.675273 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60657b3-a2c4-4626-ab0a-971262859a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:29.675967 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60657b3-a2c4-4626-ab0a-971262859a51 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:29.676280 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60657b3-a2c4-4626-ab0a-971262859a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:29.676696 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60657b3-a2c4-4626-ab0a-971262859a51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:29.677386 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60657b3-a2c4-4626-ab0a-971262859a51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:29.679448 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60657b3-a2c4-4626-ab0a-971262859a51 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:29.709506 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1051/2130] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:29 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:29.710722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4bb544-ab62-4e31-a5bd-13613320b701 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:29.720596 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4bb544-ab62-4e31-a5bd-13613320b701 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:29.767856 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:29.782636 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a7833-ae46-4a1f-9d19-2f752d4caa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:29.783462 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a7833-ae46-4a1f-9d19-2f752d4caa34 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:29.784069 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a7833-ae46-4a1f-9d19-2f752d4caa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:29.784662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a7833-ae46-4a1f-9d19-2f752d4caa34 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:29.785626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a7833-ae46-4a1f-9d19-2f752d4caa34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:29.788055 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a7833-ae46-4a1f-9d19-2f752d4caa34 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:29.811634 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1080/2131] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:29 2022] POST /identity/v3/users => generated 284 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:07:29.815635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60657b3-a2c4-4626-ab0a-971262859a51 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:29.826785 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60657b3-a2c4-4626-ab0a-971262859a51 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:29.832119 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e50883-d329-4ebb-bdde-665f45502782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:29.832930 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e50883-d329-4ebb-bdde-665f45502782 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:29.833291 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e50883-d329-4ebb-bdde-665f45502782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:29.833681 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e50883-d329-4ebb-bdde-665f45502782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:29.834765 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e50883-d329-4ebb-bdde-665f45502782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:29.836602 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e50883-d329-4ebb-bdde-665f45502782 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:29.848805 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1052/2132] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:29.854835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a7833-ae46-4a1f-9d19-2f752d4caa34 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:29.869096 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a7833-ae46-4a1f-9d19-2f752d4caa34 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:29.874500 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d026aa-8834-4f72-84a7-2d276f22ec8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:29.875328 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d026aa-8834-4f72-84a7-2d276f22ec8d admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:29.875609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d026aa-8834-4f72-84a7-2d276f22ec8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:29.875947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d026aa-8834-4f72-84a7-2d276f22ec8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:29.876848 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d026aa-8834-4f72-84a7-2d276f22ec8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:29.878657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d026aa-8834-4f72-84a7-2d276f22ec8d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:29.890991 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1081/2133] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:29.895508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e50883-d329-4ebb-bdde-665f45502782 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:29.907031 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e50883-d329-4ebb-bdde-665f45502782 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:29.915656 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d6914-d390-4957-bef4-842bd131c49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:29.916783 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d6914-d390-4957-bef4-842bd131c49e admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:29.917119 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d6914-d390-4957-bef4-842bd131c49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:29.917449 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d6914-d390-4957-bef4-842bd131c49e admin admin] PATH_INFO: `/v3/projects/f0e7cd36281c4c54a0e7a9033bb22782/users/b65ef8f2ca9446779a2f298b218ce1f9/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:29.934882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d6914-d390-4957-bef4-842bd131c49e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e7cd36281c4c54a0e7a9033bb22782, user_id=b65ef8f2ca9446779a2f298b218ce1f9, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:29.937828 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d6914-d390-4957-bef4-842bd131c49e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:29.952708 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1053/2134] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:29 2022] PUT /identity/v3/projects/f0e7cd36281c4c54a0e7a9033bb22782/users/b65ef8f2ca9446779a2f298b218ce1f9/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:29.962655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca6a89-fd6a-44ed-89a1-fcd20f49b31c None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:29.962953 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca6a89-fd6a-44ed-89a1-fcd20f49b31c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:29.963434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca6a89-fd6a-44ed-89a1-fcd20f49b31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:30.006957 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e5ca6a89-fd6a-44ed-89a1-fcd20f49b31c None None] MFA Rules not processed for user `b65ef8f2ca9446779a2f298b218ce1f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:30.008021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e5ca6a89-fd6a-44ed-89a1-fcd20f49b31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:30.111719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1082/2135] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:29 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:07:30.116492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d6914-d390-4957-bef4-842bd131c49e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:30.258520 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d6914-d390-4957-bef4-842bd131c49e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:30.281017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c1bbc-24dc-4f42-b49c-16e75308a364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:30.282212 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c1bbc-24dc-4f42-b49c-16e75308a364 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:30.282525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c1bbc-24dc-4f42-b49c-16e75308a364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:30.282980 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c1bbc-24dc-4f42-b49c-16e75308a364 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:30.283796 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c1bbc-24dc-4f42-b49c-16e75308a364 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:30.287199 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c1bbc-24dc-4f42-b49c-16e75308a364 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:30.356385 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1054/2136] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:30 2022] POST /identity/v3/projects => generated 367 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:30.360167 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca6a89-fd6a-44ed-89a1-fcd20f49b31c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:30.377886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca6a89-fd6a-44ed-89a1-fcd20f49b31c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:30.532477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae98fc2-641a-4f5e-9e72-9612cea39039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:30.533222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae98fc2-641a-4f5e-9e72-9612cea39039 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:30.533581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae98fc2-641a-4f5e-9e72-9612cea39039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:30.533698 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae98fc2-641a-4f5e-9e72-9612cea39039 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:30.534464 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae98fc2-641a-4f5e-9e72-9612cea39039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:30.536776 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae98fc2-641a-4f5e-9e72-9612cea39039 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:30.576628 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1083/2137] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:30 2022] POST /identity/v3/users => generated 290 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:30.580655 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c1bbc-24dc-4f42-b49c-16e75308a364 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:30.595251 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c1bbc-24dc-4f42-b49c-16e75308a364 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:30.601135 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7a622-5a49-4412-8d6a-7f09e6bc6e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:30.602122 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7a622-5a49-4412-8d6a-7f09e6bc6e82 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:30.602562 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7a622-5a49-4412-8d6a-7f09e6bc6e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:30.602996 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7a622-5a49-4412-8d6a-7f09e6bc6e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:30.603857 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7a622-5a49-4412-8d6a-7f09e6bc6e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:30.605657 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7a622-5a49-4412-8d6a-7f09e6bc6e82 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:30.618072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1055/2138] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:07:30.622422 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae98fc2-641a-4f5e-9e72-9612cea39039 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:30.650041 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae98fc2-641a-4f5e-9e72-9612cea39039 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:30.662308 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a98ab-f45b-4b31-b8ba-5c3be46d838e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:30.671023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a98ab-f45b-4b31-b8ba-5c3be46d838e admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:30.671234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a98ab-f45b-4b31-b8ba-5c3be46d838e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:30.671506 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a98ab-f45b-4b31-b8ba-5c3be46d838e admin admin] PATH_INFO: `/v3/projects/72d881a31a804abc9385c9d1cb94dd16/users/bbc86f38ddde48eeb5e16dc2428e230d/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:30.695220 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a98ab-f45b-4b31-b8ba-5c3be46d838e admin admin] RBAC: Authorizing `identity:create_grant(project_id=72d881a31a804abc9385c9d1cb94dd16, user_id=bbc86f38ddde48eeb5e16dc2428e230d, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:30.697497 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a98ab-f45b-4b31-b8ba-5c3be46d838e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:30.725559 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7a622-5a49-4412-8d6a-7f09e6bc6e82 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:30.726305 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1084/2139] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:30 2022] PUT /identity/v3/projects/72d881a31a804abc9385c9d1cb94dd16/users/bbc86f38ddde48eeb5e16dc2428e230d/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:07:30.738360 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7a622-5a49-4412-8d6a-7f09e6bc6e82 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:30.754410 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:30.848962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3a51c-338a-4eb7-aa29-ff009722885d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:30.850926 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3a51c-338a-4eb7-aa29-ff009722885d admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:30.851373 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3a51c-338a-4eb7-aa29-ff009722885d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:30.852879 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3a51c-338a-4eb7-aa29-ff009722885d admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:30.855194 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3a51c-338a-4eb7-aa29-ff009722885d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:30.858833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc3a51c-338a-4eb7-aa29-ff009722885d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:30.875795 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1056/2140] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:30.880702 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a98ab-f45b-4b31-b8ba-5c3be46d838e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:30.897770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a98ab-f45b-4b31-b8ba-5c3be46d838e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:30.902741 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637ff36-e3da-45ac-889f-11b7771f947c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:30.903725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637ff36-e3da-45ac-889f-11b7771f947c admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:30.904125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637ff36-e3da-45ac-889f-11b7771f947c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:30.904377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637ff36-e3da-45ac-889f-11b7771f947c admin admin] PATH_INFO: `/v3/projects/72d881a31a804abc9385c9d1cb94dd16/users/bbc86f38ddde48eeb5e16dc2428e230d/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:30.906872 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637ff36-e3da-45ac-889f-11b7771f947c admin admin] RBAC: Authorizing `identity:create_grant(project_id=72d881a31a804abc9385c9d1cb94dd16, user_id=bbc86f38ddde48eeb5e16dc2428e230d, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:30.909835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637ff36-e3da-45ac-889f-11b7771f947c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:30.921502 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1085/2141] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:30 2022] PUT /identity/v3/projects/72d881a31a804abc9385c9d1cb94dd16/users/bbc86f38ddde48eeb5e16dc2428e230d/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:30.930438 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3a51c-338a-4eb7-aa29-ff009722885d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:30.932790 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8dc3a51c-338a-4eb7-aa29-ff009722885d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:30.932993 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29636e09-5648-4348-91fa-6f166056842d None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:30.933479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29636e09-5648-4348-91fa-6f166056842d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:30.933788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29636e09-5648-4348-91fa-6f166056842d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:30.946377 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3a51c-338a-4eb7-aa29-ff009722885d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:30.955145 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:30.965477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-29636e09-5648-4348-91fa-6f166056842d None None] MFA Rules not processed for user `bbc86f38ddde48eeb5e16dc2428e230d`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:30.967731 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-29636e09-5648-4348-91fa-6f166056842d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:31.013111 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888697c-b1f3-4ced-acd3-38ba6b769b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:31.014005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888697c-b1f3-4ced-acd3-38ba6b769b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:31.014284 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888697c-b1f3-4ced-acd3-38ba6b769b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:31.014560 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888697c-b1f3-4ced-acd3-38ba6b769b25 admin admin] PATH_INFO: `/v3/users/357bcb1f9da4403582ec77ec2ca27e70` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:31.016883 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d888697c-b1f3-4ced-acd3-38ba6b769b25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=357bcb1f9da4403582ec77ec2ca27e70)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:31.019054 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d888697c-b1f3-4ced-acd3-38ba6b769b25 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:31.048140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1086/2142] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:30 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:31.061511 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29636e09-5648-4348-91fa-6f166056842d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:31.063021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-29636e09-5648-4348-91fa-6f166056842d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:31.085738 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d888697c-b1f3-4ced-acd3-38ba6b769b25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '357bcb1f9da4403582ec77ec2ca27e70'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:31.087096 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d888697c-b1f3-4ced-acd3-38ba6b769b25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '357bcb1f9da4403582ec77ec2ca27e70'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:31.098391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d888697c-b1f3-4ced-acd3-38ba6b769b25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '357bcb1f9da4403582ec77ec2ca27e70'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:31.109569 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d888697c-b1f3-4ced-acd3-38ba6b769b25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '357bcb1f9da4403582ec77ec2ca27e70'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:31.110813 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d888697c-b1f3-4ced-acd3-38ba6b769b25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '357bcb1f9da4403582ec77ec2ca27e70'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:31.125688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29636e09-5648-4348-91fa-6f166056842d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:31.130303 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f269bed-7de4-439b-8478-94be8c2e3f54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:31.131018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f269bed-7de4-439b-8478-94be8c2e3f54 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:31.131484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f269bed-7de4-439b-8478-94be8c2e3f54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:31.131767 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f269bed-7de4-439b-8478-94be8c2e3f54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:31.133558 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1057/2143] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:30 2022] DELETE /identity/v3/users/357bcb1f9da4403582ec77ec2ca27e70 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:31.134318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0f269bed-7de4-439b-8478-94be8c2e3f54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:31.138004 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c05fc8-2566-4bd1-b5e6-e5b488b1ab7b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:31.138292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c05fc8-2566-4bd1-b5e6-e5b488b1ab7b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:31.138573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c05fc8-2566-4bd1-b5e6-e5b488b1ab7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:31.148681 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f269bed-7de4-439b-8478-94be8c2e3f54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:31.150044 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f269bed-7de4-439b-8478-94be8c2e3f54 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:31.159365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-30c05fc8-2566-4bd1-b5e6-e5b488b1ab7b None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:31.160651 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-30c05fc8-2566-4bd1-b5e6-e5b488b1ab7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:31.168551 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1087/2144] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:31 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:07:31.200606 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:31.253276 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1058/2145] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:31.263297 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f269bed-7de4-439b-8478-94be8c2e3f54 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:31.264826 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0f269bed-7de4-439b-8478-94be8c2e3f54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:31.270461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c05fc8-2566-4bd1-b5e6-e5b488b1ab7b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:31.272338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-30c05fc8-2566-4bd1-b5e6-e5b488b1ab7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:31.334961 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f269bed-7de4-439b-8478-94be8c2e3f54 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:31.335182 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c05fc8-2566-4bd1-b5e6-e5b488b1ab7b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:31.338820 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472b0ff-151b-4d56-b490-9981255eefb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:31.339481 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472b0ff-151b-4d56-b490-9981255eefb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:31.339764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472b0ff-151b-4d56-b490-9981255eefb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:31.340256 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472b0ff-151b-4d56-b490-9981255eefb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:31.342730 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d472b0ff-151b-4d56-b490-9981255eefb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:31.406590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8866d5b-2d27-44bd-b456-00d20bd4ae8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:31.407550 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8866d5b-2d27-44bd-b456-00d20bd4ae8c service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:31.407908 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8866d5b-2d27-44bd-b456-00d20bd4ae8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:31.408258 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8866d5b-2d27-44bd-b456-00d20bd4ae8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:31.420099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d472b0ff-151b-4d56-b490-9981255eefb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:31.421966 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d472b0ff-151b-4d56-b490-9981255eefb2 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:31.433371 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1059/2146] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:31 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:31.439219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:31.444648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8866d5b-2d27-44bd-b456-00d20bd4ae8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:31.446318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8866d5b-2d27-44bd-b456-00d20bd4ae8c service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:31.470243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1088/2147] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:31.931406 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472b0ff-151b-4d56-b490-9981255eefb2 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:31.941430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472b0ff-151b-4d56-b490-9981255eefb2 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:31.946303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98b950-b1e1-46e2-ad1b-386645364602 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:31.947131 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98b950-b1e1-46e2-ad1b-386645364602 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:31.947438 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98b950-b1e1-46e2-ad1b-386645364602 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:31.947716 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98b950-b1e1-46e2-ad1b-386645364602 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:31.955034 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ab98b950-b1e1-46e2-ad1b-386645364602 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:32.058840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98b950-b1e1-46e2-ad1b-386645364602 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:32.060643 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98b950-b1e1-46e2-ad1b-386645364602 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:32.087392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1060/2148] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:32.209433 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8866d5b-2d27-44bd-b456-00d20bd4ae8c service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:32.211172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c8866d5b-2d27-44bd-b456-00d20bd4ae8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:32.247562 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8866d5b-2d27-44bd-b456-00d20bd4ae8c service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:32.266734 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc54c4-9bbd-4bb3-992c-d3bcfd0a354b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:32.267794 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc54c4-9bbd-4bb3-992c-d3bcfd0a354b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:32.268056 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc54c4-9bbd-4bb3-992c-d3bcfd0a354b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:32.268335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc54c4-9bbd-4bb3-992c-d3bcfd0a354b admin admin] PATH_INFO: `/v3/projects/ca0907430f404357b91d31c57ff07b34` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:32.269804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc54c4-9bbd-4bb3-992c-d3bcfd0a354b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca0907430f404357b91d31c57ff07b34)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:32.273154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc54c4-9bbd-4bb3-992c-d3bcfd0a354b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:32.380492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-29dc54c4-9bbd-4bb3-992c-d3bcfd0a354b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca0907430f404357b91d31c57ff07b34', 'request_id': 'req-29dc54c4-9bbd-4bb3-992c-d3bcfd0a354b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:32.399229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-29dc54c4-9bbd-4bb3-992c-d3bcfd0a354b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca0907430f404357b91d31c57ff07b34', 'request_id': 'req-29dc54c4-9bbd-4bb3-992c-d3bcfd0a354b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:32.416222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-29dc54c4-9bbd-4bb3-992c-d3bcfd0a354b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca0907430f404357b91d31c57ff07b34', 'request_id': 'req-29dc54c4-9bbd-4bb3-992c-d3bcfd0a354b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:32.441036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-29dc54c4-9bbd-4bb3-992c-d3bcfd0a354b admin admin] The token cache is being invalidate because project ca0907430f404357b91d31c57ff07b34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:32.441619 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-29dc54c4-9bbd-4bb3-992c-d3bcfd0a354b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:32.442021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-29dc54c4-9bbd-4bb3-992c-d3bcfd0a354b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:32.444006 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1089/2149] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:32 2022] DELETE /identity/v3/projects/ca0907430f404357b91d31c57ff07b34 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:07:32.448115 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98b950-b1e1-46e2-ad1b-386645364602 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:32.470802 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98b950-b1e1-46e2-ad1b-386645364602 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:32.509705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:32.588905 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa96e77-2b56-4cf2-9af2-42b72305d848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:32.590437 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa96e77-2b56-4cf2-9af2-42b72305d848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:32.590683 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa96e77-2b56-4cf2-9af2-42b72305d848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:32.590956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa96e77-2b56-4cf2-9af2-42b72305d848 admin admin] PATH_INFO: `/v3/users/f668820f39d242d1bcb16b41163080cd` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:32.592356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa96e77-2b56-4cf2-9af2-42b72305d848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f668820f39d242d1bcb16b41163080cd)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:32.596859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa96e77-2b56-4cf2-9af2-42b72305d848 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:32.680745 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5fa96e77-2b56-4cf2-9af2-42b72305d848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f668820f39d242d1bcb16b41163080cd'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:32.681586 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5fa96e77-2b56-4cf2-9af2-42b72305d848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f668820f39d242d1bcb16b41163080cd'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:32.691972 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5fa96e77-2b56-4cf2-9af2-42b72305d848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f668820f39d242d1bcb16b41163080cd'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:32.701466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5fa96e77-2b56-4cf2-9af2-42b72305d848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f668820f39d242d1bcb16b41163080cd'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:32.702248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5fa96e77-2b56-4cf2-9af2-42b72305d848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f668820f39d242d1bcb16b41163080cd'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:32.721647 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1061/2150] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:32 2022] DELETE /identity/v3/users/f668820f39d242d1bcb16b41163080cd => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:33.277349 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc54c4-9bbd-4bb3-992c-d3bcfd0a354b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:33.282229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-29dc54c4-9bbd-4bb3-992c-d3bcfd0a354b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:33.319697 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc54c4-9bbd-4bb3-992c-d3bcfd0a354b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:33.444119 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbb84f-37bf-40b7-92c7-5f2baa40128d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:33.445148 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbb84f-37bf-40b7-92c7-5f2baa40128d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:33.445471 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbb84f-37bf-40b7-92c7-5f2baa40128d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:33.445751 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbb84f-37bf-40b7-92c7-5f2baa40128d admin admin] PATH_INFO: `/v3/projects/bf4435a5ccf848be9cd4b6e9dc1c269c` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:33.447007 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbb84f-37bf-40b7-92c7-5f2baa40128d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf4435a5ccf848be9cd4b6e9dc1c269c)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:33.454357 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbb84f-37bf-40b7-92c7-5f2baa40128d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:33.562483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d0dbb84f-37bf-40b7-92c7-5f2baa40128d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf4435a5ccf848be9cd4b6e9dc1c269c', 'request_id': 'req-d0dbb84f-37bf-40b7-92c7-5f2baa40128d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:33.569630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d0dbb84f-37bf-40b7-92c7-5f2baa40128d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf4435a5ccf848be9cd4b6e9dc1c269c', 'request_id': 'req-d0dbb84f-37bf-40b7-92c7-5f2baa40128d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:33.578106 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d0dbb84f-37bf-40b7-92c7-5f2baa40128d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf4435a5ccf848be9cd4b6e9dc1c269c', 'request_id': 'req-d0dbb84f-37bf-40b7-92c7-5f2baa40128d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:33.610577 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d0dbb84f-37bf-40b7-92c7-5f2baa40128d admin admin] The token cache is being invalidate because project bf4435a5ccf848be9cd4b6e9dc1c269c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:33.610919 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d0dbb84f-37bf-40b7-92c7-5f2baa40128d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:33.611617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d0dbb84f-37bf-40b7-92c7-5f2baa40128d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:33.617309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1090/2151] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:33 2022] DELETE /identity/v3/projects/bf4435a5ccf848be9cd4b6e9dc1c269c => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:33.619532 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a19f92-1a17-4b53-86ac-5b0be02335fc None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:33.619532 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a19f92-1a17-4b53-86ac-5b0be02335fc None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:33.620077 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a19f92-1a17-4b53-86ac-5b0be02335fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:33.657268 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b1a19f92-1a17-4b53-86ac-5b0be02335fc None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:33.658594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b1a19f92-1a17-4b53-86ac-5b0be02335fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:33.774486 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1062/2152] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:33.782390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9307ab-2176-4193-ae2a-9a58190806cf None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:33.783571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9307ab-2176-4193-ae2a-9a58190806cf None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:33.783750 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9307ab-2176-4193-ae2a-9a58190806cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:33.833451 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-aa9307ab-2176-4193-ae2a-9a58190806cf None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:33.834917 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-aa9307ab-2176-4193-ae2a-9a58190806cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:33.843589 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1091/2153] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:33.848345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a19f92-1a17-4b53-86ac-5b0be02335fc None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:33.870927 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a19f92-1a17-4b53-86ac-5b0be02335fc None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:33.885445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44002d0f-669f-43af-9842-5abd02b61a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:33.888045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44002d0f-669f-43af-9842-5abd02b61a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:33.889785 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44002d0f-669f-43af-9842-5abd02b61a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:33.890379 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44002d0f-669f-43af-9842-5abd02b61a3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:33.891256 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44002d0f-669f-43af-9842-5abd02b61a3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:07:33.893840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44002d0f-669f-43af-9842-5abd02b61a3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:33.897456 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44002d0f-669f-43af-9842-5abd02b61a3f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:33.937852 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1063/2154] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:07:33.942198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9307ab-2176-4193-ae2a-9a58190806cf None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:33.947204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:33.956697 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9307ab-2176-4193-ae2a-9a58190806cf None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:33.965539 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a03e3a-1519-4743-9733-d4e9c398d3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:33.966799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a03e3a-1519-4743-9733-d4e9c398d3e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:33.967424 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a03e3a-1519-4743-9733-d4e9c398d3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:33.967861 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a03e3a-1519-4743-9733-d4e9c398d3e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:33.968784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a03e3a-1519-4743-9733-d4e9c398d3e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:33.971063 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a03e3a-1519-4743-9733-d4e9c398d3e8 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:34.017859 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1092/2155] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:33 2022] POST /identity/v3/projects => generated 387 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:34.018310 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44002d0f-669f-43af-9842-5abd02b61a3f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:34.031280 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44002d0f-669f-43af-9842-5abd02b61a3f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:34.083247 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:34.158221 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f6409-f0ea-4b51-9838-fcc7a8a81625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:34.159011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f6409-f0ea-4b51-9838-fcc7a8a81625 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:34.159294 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f6409-f0ea-4b51-9838-fcc7a8a81625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:34.160107 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f6409-f0ea-4b51-9838-fcc7a8a81625 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:34.160376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f6409-f0ea-4b51-9838-fcc7a8a81625 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:34.163717 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f6409-f0ea-4b51-9838-fcc7a8a81625 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:34.191026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1064/2156] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:34 2022] POST /identity/v3/users => generated 294 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:07:34.196444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a03e3a-1519-4743-9733-d4e9c398d3e8 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:34.210847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f6409-f0ea-4b51-9838-fcc7a8a81625 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:34.212918 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1e4f6409-f0ea-4b51-9838-fcc7a8a81625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:34.214546 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a03e3a-1519-4743-9733-d4e9c398d3e8 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:34.220882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037eb2e9-ee82-409f-afd1-c1f61b574ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:34.221989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037eb2e9-ee82-409f-afd1-c1f61b574ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:34.222476 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037eb2e9-ee82-409f-afd1-c1f61b574ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:34.223140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037eb2e9-ee82-409f-afd1-c1f61b574ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:34.224117 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037eb2e9-ee82-409f-afd1-c1f61b574ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:34.226665 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037eb2e9-ee82-409f-afd1-c1f61b574ce3 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:34.238762 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1093/2157] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:34.241334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f6409-f0ea-4b51-9838-fcc7a8a81625 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:34.242779 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037eb2e9-ee82-409f-afd1-c1f61b574ce3 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:34.254163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037eb2e9-ee82-409f-afd1-c1f61b574ce3 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:34.254758 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214a92f-fdb5-4774-be90-874c6b54ff5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:34.257076 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214a92f-fdb5-4774-be90-874c6b54ff5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:34.257406 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214a92f-fdb5-4774-be90-874c6b54ff5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:34.257682 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214a92f-fdb5-4774-be90-874c6b54ff5a admin admin] PATH_INFO: `/v3/users/be06179595ef450a917d71fe4e6b2c5b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:34.259013 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214a92f-fdb5-4774-be90-874c6b54ff5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=be06179595ef450a917d71fe4e6b2c5b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:34.260226 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7412aa-0a8d-402b-a628-2d34e92b7794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:34.261013 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7412aa-0a8d-402b-a628-2d34e92b7794 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:34.261300 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7412aa-0a8d-402b-a628-2d34e92b7794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:34.261730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7412aa-0a8d-402b-a628-2d34e92b7794 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:34.262092 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214a92f-fdb5-4774-be90-874c6b54ff5a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:34.262493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7412aa-0a8d-402b-a628-2d34e92b7794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:34.264191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7412aa-0a8d-402b-a628-2d34e92b7794 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:34.278940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1094/2158] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:07:34.283164 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7412aa-0a8d-402b-a628-2d34e92b7794 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:34.295607 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7412aa-0a8d-402b-a628-2d34e92b7794 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:34.302669 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a8374-6a38-4e89-b601-37fb2d6d80c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:34.304376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a8374-6a38-4e89-b601-37fb2d6d80c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:34.304787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a8374-6a38-4e89-b601-37fb2d6d80c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:34.305132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a8374-6a38-4e89-b601-37fb2d6d80c7 admin admin] PATH_INFO: `/v3/projects/ae0ab5bafa0547738a0214a2f0fb750b/users/56a897b3156b44829d9b954b6f5224e8/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:34.326401 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a8374-6a38-4e89-b601-37fb2d6d80c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae0ab5bafa0547738a0214a2f0fb750b, user_id=56a897b3156b44829d9b954b6f5224e8, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:34.329216 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a8374-6a38-4e89-b601-37fb2d6d80c7 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:34.336828 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-2214a92f-fdb5-4774-be90-874c6b54ff5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be06179595ef450a917d71fe4e6b2c5b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:34.337653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-2214a92f-fdb5-4774-be90-874c6b54ff5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be06179595ef450a917d71fe4e6b2c5b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:34.354057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-2214a92f-fdb5-4774-be90-874c6b54ff5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be06179595ef450a917d71fe4e6b2c5b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:34.360465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1095/2159] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:34 2022] PUT /identity/v3/projects/ae0ab5bafa0547738a0214a2f0fb750b/users/56a897b3156b44829d9b954b6f5224e8/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:34.366946 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09dd957-371a-4171-8736-90ba7f627148 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:34.367272 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09dd957-371a-4171-8736-90ba7f627148 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:34.367553 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09dd957-371a-4171-8736-90ba7f627148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:34.370064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-2214a92f-fdb5-4774-be90-874c6b54ff5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be06179595ef450a917d71fe4e6b2c5b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:34.370832 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-2214a92f-fdb5-4774-be90-874c6b54ff5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be06179595ef450a917d71fe4e6b2c5b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:34.408723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1065/2160] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:34 2022] DELETE /identity/v3/users/be06179595ef450a917d71fe4e6b2c5b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:34.414755 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-d09dd957-371a-4171-8736-90ba7f627148 None None] MFA Rules not processed for user `56a897b3156b44829d9b954b6f5224e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:34.416066 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d09dd957-371a-4171-8736-90ba7f627148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:34.508936 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1096/2161] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:34.513871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214a92f-fdb5-4774-be90-874c6b54ff5a admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:34.515664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2214a92f-fdb5-4774-be90-874c6b54ff5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:34.618754 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214a92f-fdb5-4774-be90-874c6b54ff5a admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:34.634499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bb419-93ce-4489-8508-f21b65a30a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:34.635372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bb419-93ce-4489-8508-f21b65a30a9a admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:34.635660 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bb419-93ce-4489-8508-f21b65a30a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:34.635931 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bb419-93ce-4489-8508-f21b65a30a9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:34.636692 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8bb419-93ce-4489-8508-f21b65a30a9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:34.638634 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8bb419-93ce-4489-8508-f21b65a30a9a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:34.684024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1066/2162] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:34 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:34.685412 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09dd957-371a-4171-8736-90ba7f627148 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:34.696797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bb419-93ce-4489-8508-f21b65a30a9a admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:34.698913 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-0e8bb419-93ce-4489-8508-f21b65a30a9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:34.701268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09dd957-371a-4171-8736-90ba7f627148 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:34.737490 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:34.807922 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bb419-93ce-4489-8508-f21b65a30a9a admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:34.813810 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e4a71-cd26-4b26-875e-1d22c6f2b005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:34.814778 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e4a71-cd26-4b26-875e-1d22c6f2b005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:34.815135 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e4a71-cd26-4b26-875e-1d22c6f2b005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:34.815613 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e4a71-cd26-4b26-875e-1d22c6f2b005 admin admin] PATH_INFO: `/v3/projects/c348649c6b794f5c9339ea9506b6a16b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:34.815727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb774c3-b92e-461b-87c3-8c2b6f1cf389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:34.817217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e4a71-cd26-4b26-875e-1d22c6f2b005 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c348649c6b794f5c9339ea9506b6a16b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:34.817721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb774c3-b92e-461b-87c3-8c2b6f1cf389 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:34.818016 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb774c3-b92e-461b-87c3-8c2b6f1cf389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:34.818331 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb774c3-b92e-461b-87c3-8c2b6f1cf389 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:34.819302 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb774c3-b92e-461b-87c3-8c2b6f1cf389 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:34.819414 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e4a71-cd26-4b26-875e-1d22c6f2b005 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:34.822476 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb774c3-b92e-461b-87c3-8c2b6f1cf389 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:34.860074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1097/2163] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:34 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:34.861236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb774c3-b92e-461b-87c3-8c2b6f1cf389 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:34.874741 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb774c3-b92e-461b-87c3-8c2b6f1cf389 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:34.887778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:34.892669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-dd3e4a71-cd26-4b26-875e-1d22c6f2b005 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c348649c6b794f5c9339ea9506b6a16b', 'request_id': 'req-dd3e4a71-cd26-4b26-875e-1d22c6f2b005'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:34.901969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-dd3e4a71-cd26-4b26-875e-1d22c6f2b005 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c348649c6b794f5c9339ea9506b6a16b', 'request_id': 'req-dd3e4a71-cd26-4b26-875e-1d22c6f2b005'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:34.909682 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-dd3e4a71-cd26-4b26-875e-1d22c6f2b005 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c348649c6b794f5c9339ea9506b6a16b', 'request_id': 'req-dd3e4a71-cd26-4b26-875e-1d22c6f2b005'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:34.933706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-dd3e4a71-cd26-4b26-875e-1d22c6f2b005 admin admin] The token cache is being invalidate because project c348649c6b794f5c9339ea9506b6a16b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:34.934000 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-dd3e4a71-cd26-4b26-875e-1d22c6f2b005 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:34.934839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-dd3e4a71-cd26-4b26-875e-1d22c6f2b005 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:34.936846 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1067/2164] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:34 2022] DELETE /identity/v3/projects/c348649c6b794f5c9339ea9506b6a16b => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:07:34.941908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576e5a0-4dff-45ed-adbe-4c8b8fd29720 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:34.942220 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576e5a0-4dff-45ed-adbe-4c8b8fd29720 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:34.942651 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576e5a0-4dff-45ed-adbe-4c8b8fd29720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:34.950384 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff27523-bbda-465b-8ef8-aad3242ef464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:34.951526 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff27523-bbda-465b-8ef8-aad3242ef464 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:34.951911 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff27523-bbda-465b-8ef8-aad3242ef464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:34.952388 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff27523-bbda-465b-8ef8-aad3242ef464 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:34.953417 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff27523-bbda-465b-8ef8-aad3242ef464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:34.955833 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff27523-bbda-465b-8ef8-aad3242ef464 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:34.963198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5576e5a0-4dff-45ed-adbe-4c8b8fd29720 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:34.964748 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5576e5a0-4dff-45ed-adbe-4c8b8fd29720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:34.972585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1098/2165] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:34.973750 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff27523-bbda-465b-8ef8-aad3242ef464 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:34.975971 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-aff27523-bbda-465b-8ef8-aad3242ef464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:34.977701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1068/2166] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:07:34.986321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc7456-f0d6-461e-a738-7417628068f0 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:34.986819 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc7456-f0d6-461e-a738-7417628068f0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:34.987200 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc7456-f0d6-461e-a738-7417628068f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:34.998273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff27523-bbda-465b-8ef8-aad3242ef464 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:35.004122 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4e684-eb2d-44e6-92b2-ca1ce5b8e8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:35.004866 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4e684-eb2d-44e6-92b2-ca1ce5b8e8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:35.005206 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4e684-eb2d-44e6-92b2-ca1ce5b8e8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:35.005793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4e684-eb2d-44e6-92b2-ca1ce5b8e8e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:35.006402 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4e684-eb2d-44e6-92b2-ca1ce5b8e8e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:35.008007 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4e684-eb2d-44e6-92b2-ca1ce5b8e8e2 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:35.020212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1099/2167] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:07:35.024777 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4e684-eb2d-44e6-92b2-ca1ce5b8e8e2 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:35.029369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c9bc7456-f0d6-461e-a738-7417628068f0 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:35.032605 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c9bc7456-f0d6-461e-a738-7417628068f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:35.036998 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4e684-eb2d-44e6-92b2-ca1ce5b8e8e2 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:35.043632 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ed256-c0e7-4f73-9ebc-ac216a8fccbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:35.044667 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ed256-c0e7-4f73-9ebc-ac216a8fccbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:35.045001 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ed256-c0e7-4f73-9ebc-ac216a8fccbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:35.045338 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ed256-c0e7-4f73-9ebc-ac216a8fccbd admin admin] PATH_INFO: `/v3/projects/3fab615f4c894dc8927dd9be1f30dcd6/users/a8b099bf715e40948d9def8f7e43f3b0/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:35.049990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1069/2168] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:35.054319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc7456-f0d6-461e-a738-7417628068f0 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:35.062220 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ed256-c0e7-4f73-9ebc-ac216a8fccbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fab615f4c894dc8927dd9be1f30dcd6, user_id=a8b099bf715e40948d9def8f7e43f3b0, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:35.066937 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ed256-c0e7-4f73-9ebc-ac216a8fccbd admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:35.068575 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc7456-f0d6-461e-a738-7417628068f0 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:35.075724 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12247a8-8e3d-4c3a-b746-5b60b8252da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:35.076457 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12247a8-8e3d-4c3a-b746-5b60b8252da7 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:35.076815 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12247a8-8e3d-4c3a-b746-5b60b8252da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:35.077019 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12247a8-8e3d-4c3a-b746-5b60b8252da7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:35.077752 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12247a8-8e3d-4c3a-b746-5b60b8252da7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:07:35.083958 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12247a8-8e3d-4c3a-b746-5b60b8252da7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:35.083958 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12247a8-8e3d-4c3a-b746-5b60b8252da7 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:35.090214 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1100/2169] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:35 2022] PUT /identity/v3/projects/3fab615f4c894dc8927dd9be1f30dcd6/users/a8b099bf715e40948d9def8f7e43f3b0/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:07:35.092452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1070/2170] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:07:35.099889 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ed256-c0e7-4f73-9ebc-ac216a8fccbd admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:35.100557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66411176-cb81-4fec-8f95-65eccb153e19 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:35.101039 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66411176-cb81-4fec-8f95-65eccb153e19 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:35.101464 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66411176-cb81-4fec-8f95-65eccb153e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:35.150556 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-66411176-cb81-4fec-8f95-65eccb153e19 None None] MFA Rules not processed for user `a8b099bf715e40948d9def8f7e43f3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:35.151877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-66411176-cb81-4fec-8f95-65eccb153e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:35.183096 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ed256-c0e7-4f73-9ebc-ac216a8fccbd admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:35.191253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ca076-0f21-48d4-8700-45dc78b22aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:35.191253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ca076-0f21-48d4-8700-45dc78b22aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:35.191542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ca076-0f21-48d4-8700-45dc78b22aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:35.191810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ca076-0f21-48d4-8700-45dc78b22aa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:35.196809 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ca076-0f21-48d4-8700-45dc78b22aa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:35.198842 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ca076-0f21-48d4-8700-45dc78b22aa5 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:35.236819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1101/2171] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:35 2022] POST /identity/v3/projects => generated 389 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:35.238796 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ca076-0f21-48d4-8700-45dc78b22aa5 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:35.304700 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1071/2172] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:35.305686 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66411176-cb81-4fec-8f95-65eccb153e19 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:35.317460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66411176-cb81-4fec-8f95-65eccb153e19 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:35.322052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:35.342398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ca076-0f21-48d4-8700-45dc78b22aa5 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:35.347439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1373e-f5b8-4d40-a8f3-1566314b0925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:35.348206 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1373e-f5b8-4d40-a8f3-1566314b0925 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:35.348713 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1373e-f5b8-4d40-a8f3-1566314b0925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:35.349057 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1373e-f5b8-4d40-a8f3-1566314b0925 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:35.350728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e1373e-f5b8-4d40-a8f3-1566314b0925 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:35.352928 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e1373e-f5b8-4d40-a8f3-1566314b0925 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:35.368739 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:35.389389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1102/2173] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:35 2022] POST /identity/v3/users => generated 295 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:35.393949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1373e-f5b8-4d40-a8f3-1566314b0925 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:35.405549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1373e-f5b8-4d40-a8f3-1566314b0925 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:35.406676 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5ee91-be2f-4751-859f-0e83e7254079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:35.407504 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5ee91-be2f-4751-859f-0e83e7254079 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:35.407787 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5ee91-be2f-4751-859f-0e83e7254079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:35.408061 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5ee91-be2f-4751-859f-0e83e7254079 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:35.408864 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5ee91-be2f-4751-859f-0e83e7254079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:35.410952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5ee91-be2f-4751-859f-0e83e7254079 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:35.411761 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b497fb8-0160-417d-bdab-da160b22437f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:35.413488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b497fb8-0160-417d-bdab-da160b22437f admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:35.413944 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b497fb8-0160-417d-bdab-da160b22437f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:35.414279 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b497fb8-0160-417d-bdab-da160b22437f admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:35.415432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b497fb8-0160-417d-bdab-da160b22437f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:35.417908 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b497fb8-0160-417d-bdab-da160b22437f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:35.429780 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1103/2174] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:35.433811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b497fb8-0160-417d-bdab-da160b22437f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:35.444582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b497fb8-0160-417d-bdab-da160b22437f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:35.452475 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1072/2175] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:35 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:07:35.452877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf751c-82f5-4b5e-aabd-9601b06c49cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:35.452877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf751c-82f5-4b5e-aabd-9601b06c49cf admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:35.452877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf751c-82f5-4b5e-aabd-9601b06c49cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:35.452877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf751c-82f5-4b5e-aabd-9601b06c49cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:35.453409 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf751c-82f5-4b5e-aabd-9601b06c49cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:35.454192 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5ee91-be2f-4751-859f-0e83e7254079 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:35.455208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf751c-82f5-4b5e-aabd-9601b06c49cf admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:35.469411 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1104/2176] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:07:35.474534 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf751c-82f5-4b5e-aabd-9601b06c49cf admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:35.487421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf751c-82f5-4b5e-aabd-9601b06c49cf admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:35.548653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5ee91-be2f-4751-859f-0e83e7254079 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:35.554176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043f303-7050-4b26-bdfc-b615c743afa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:35.555078 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043f303-7050-4b26-bdfc-b615c743afa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:35.555435 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043f303-7050-4b26-bdfc-b615c743afa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:35.555782 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043f303-7050-4b26-bdfc-b615c743afa9 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:35.556591 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043f303-7050-4b26-bdfc-b615c743afa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:35.558632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043f303-7050-4b26-bdfc-b615c743afa9 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:35.563435 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e505fa6-254b-49d6-baf9-f5daba00bff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:35.563816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e505fa6-254b-49d6-baf9-f5daba00bff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:35.564162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e505fa6-254b-49d6-baf9-f5daba00bff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:35.564535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e505fa6-254b-49d6-baf9-f5daba00bff1 admin admin] PATH_INFO: `/v3/projects/0da9a82039c24056beec4b61f533eaa2/users/744eb9d4e7084b7e9cae36d5de67f0f0/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:35.584813 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e505fa6-254b-49d6-baf9-f5daba00bff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0da9a82039c24056beec4b61f533eaa2, user_id=744eb9d4e7084b7e9cae36d5de67f0f0, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:35.587892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e505fa6-254b-49d6-baf9-f5daba00bff1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:35.592649 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1073/2177] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:35 2022] POST /identity/v3/users => generated 299 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:07:35.596481 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043f303-7050-4b26-bdfc-b615c743afa9 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:35.614405 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1105/2178] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:35 2022] PUT /identity/v3/projects/0da9a82039c24056beec4b61f533eaa2/users/744eb9d4e7084b7e9cae36d5de67f0f0/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:35.621535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07717735-bbe4-43ba-880c-00ee3e160252 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:35.621855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07717735-bbe4-43ba-880c-00ee3e160252 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:35.622162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07717735-bbe4-43ba-880c-00ee3e160252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:35.626175 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043f303-7050-4b26-bdfc-b615c743afa9 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:35.645361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-07717735-bbe4-43ba-880c-00ee3e160252 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:35.646648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-07717735-bbe4-43ba-880c-00ee3e160252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:35.728205 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1324ca3-0944-4456-9b43-0518fd7cb6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:35.729053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1324ca3-0944-4456-9b43-0518fd7cb6bd admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:35.729424 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1324ca3-0944-4456-9b43-0518fd7cb6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:35.729788 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1324ca3-0944-4456-9b43-0518fd7cb6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:35.730629 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1324ca3-0944-4456-9b43-0518fd7cb6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:35.732902 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1106/2179] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:35.733475 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1324ca3-0944-4456-9b43-0518fd7cb6bd admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:35.745715 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07717735-bbe4-43ba-880c-00ee3e160252 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:35.747571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-07717735-bbe4-43ba-880c-00ee3e160252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:35.748579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1074/2180] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:35.752494 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1324ca3-0944-4456-9b43-0518fd7cb6bd admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:35.764787 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1324ca3-0944-4456-9b43-0518fd7cb6bd admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:35.776079 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c36d0-3ee0-444f-a8e8-1812ec87cabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:35.777679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c36d0-3ee0-444f-a8e8-1812ec87cabb admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:35.777970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c36d0-3ee0-444f-a8e8-1812ec87cabb admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:35.778260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c36d0-3ee0-444f-a8e8-1812ec87cabb admin admin] PATH_INFO: `/v3/projects/cf5c5d3011f1434ba00e3f79078541bd/users/1f4120288ec44586935a754216dfac72/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:35.795363 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c36d0-3ee0-444f-a8e8-1812ec87cabb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf5c5d3011f1434ba00e3f79078541bd, user_id=1f4120288ec44586935a754216dfac72, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:35.798115 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c36d0-3ee0-444f-a8e8-1812ec87cabb admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:35.800344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:35.813689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1075/2181] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:35 2022] PUT /identity/v3/projects/cf5c5d3011f1434ba00e3f79078541bd/users/1f4120288ec44586935a754216dfac72/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:07:35.817612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c36d0-3ee0-444f-a8e8-1812ec87cabb admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:35.823870 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07717735-bbe4-43ba-880c-00ee3e160252 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:35.827873 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:35.890866 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53f44a-e991-485a-aded-1a4762d2a306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:35.892806 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53f44a-e991-485a-aded-1a4762d2a306 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:35.893112 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53f44a-e991-485a-aded-1a4762d2a306 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:35.893940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53f44a-e991-485a-aded-1a4762d2a306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:35.898775 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c36d0-3ee0-444f-a8e8-1812ec87cabb admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:35.903468 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe0e1d-ada8-4735-a790-1bbb2a38f326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:35.903468 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe0e1d-ada8-4735-a790-1bbb2a38f326 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:35.903468 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe0e1d-ada8-4735-a790-1bbb2a38f326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:35.904664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe0e1d-ada8-4735-a790-1bbb2a38f326 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:35.904821 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe0e1d-ada8-4735-a790-1bbb2a38f326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:35.907152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53f44a-e991-485a-aded-1a4762d2a306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:35.907278 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe0e1d-ada8-4735-a790-1bbb2a38f326 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:35.908721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53f44a-e991-485a-aded-1a4762d2a306 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:35.922214 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1076/2182] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:35.924301 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1107/2183] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:35.925509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe0e1d-ada8-4735-a790-1bbb2a38f326 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:35.940596 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe0e1d-ada8-4735-a790-1bbb2a38f326 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:35.949024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5ea4d-5489-4a1a-8543-c6428bdf4c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:35.950087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5ea4d-5489-4a1a-8543-c6428bdf4c3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:35.950567 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5ea4d-5489-4a1a-8543-c6428bdf4c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:35.951171 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5ea4d-5489-4a1a-8543-c6428bdf4c3e admin admin] PATH_INFO: `/v3/projects/cf5c5d3011f1434ba00e3f79078541bd/users/1f4120288ec44586935a754216dfac72/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:35.954237 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5ea4d-5489-4a1a-8543-c6428bdf4c3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf5c5d3011f1434ba00e3f79078541bd, user_id=1f4120288ec44586935a754216dfac72, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:35.959167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5ea4d-5489-4a1a-8543-c6428bdf4c3e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:35.976485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1077/2184] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:35 2022] PUT /identity/v3/projects/cf5c5d3011f1434ba00e3f79078541bd/users/1f4120288ec44586935a754216dfac72/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:35.989739 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc096e8-ce91-4d96-b3bd-42e592b80f6b None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:35.990064 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc096e8-ce91-4d96-b3bd-42e592b80f6b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:35.990485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc096e8-ce91-4d96-b3bd-42e592b80f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:36.034484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7bc096e8-ce91-4d96-b3bd-42e592b80f6b None None] MFA Rules not processed for user `1f4120288ec44586935a754216dfac72`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:36.036171 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7bc096e8-ce91-4d96-b3bd-42e592b80f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:36.128650 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1108/2185] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:36.144128 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5ea4d-5489-4a1a-8543-c6428bdf4c3e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:36.145959 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-87f5ea4d-5489-4a1a-8543-c6428bdf4c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:36.217836 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5ea4d-5489-4a1a-8543-c6428bdf4c3e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:36.221720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026405a-a258-4d44-b30c-76655fef639c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:36.222380 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026405a-a258-4d44-b30c-76655fef639c service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:36.222663 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026405a-a258-4d44-b30c-76655fef639c service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:36.222941 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026405a-a258-4d44-b30c-76655fef639c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:36.236565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a026405a-a258-4d44-b30c-76655fef639c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:36.238380 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a026405a-a258-4d44-b30c-76655fef639c service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:36.252521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1078/2186] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:36.941669 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc096e8-ce91-4d96-b3bd-42e592b80f6b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:36.946539 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7bc096e8-ce91-4d96-b3bd-42e592b80f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:36.961981 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc096e8-ce91-4d96-b3bd-42e592b80f6b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:36.966066 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ea708-6a98-497f-88f8-77764c4f63f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:36.966778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ea708-6a98-497f-88f8-77764c4f63f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:36.967058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ea708-6a98-497f-88f8-77764c4f63f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:36.967337 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ea708-6a98-497f-88f8-77764c4f63f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:36.969232 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4b7ea708-6a98-497f-88f8-77764c4f63f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:37.039227 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026405a-a258-4d44-b30c-76655fef639c service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:37.040676 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a026405a-a258-4d44-b30c-76655fef639c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:37.054916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ea708-6a98-497f-88f8-77764c4f63f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:37.057257 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ea708-6a98-497f-88f8-77764c4f63f6 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:37.068969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1109/2187] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:37.115455 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026405a-a258-4d44-b30c-76655fef639c service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:37.120982 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2797d6-328e-48f7-9a4e-89a3947064d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:37.122024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2797d6-328e-48f7-9a4e-89a3947064d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:37.122357 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2797d6-328e-48f7-9a4e-89a3947064d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:37.122714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2797d6-328e-48f7-9a4e-89a3947064d7 admin admin] PATH_INFO: `/v3/users/b65ef8f2ca9446779a2f298b218ce1f9` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:37.124053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2797d6-328e-48f7-9a4e-89a3947064d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b65ef8f2ca9446779a2f298b218ce1f9)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:37.126203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2797d6-328e-48f7-9a4e-89a3947064d7 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:37.168051 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:INSERT=2,keystone:SELECT=95,keystone:DELETE=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:37.221169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8a2797d6-328e-48f7-9a4e-89a3947064d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b65ef8f2ca9446779a2f298b218ce1f9'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:37.222168 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8a2797d6-328e-48f7-9a4e-89a3947064d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b65ef8f2ca9446779a2f298b218ce1f9'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:37.234625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8a2797d6-328e-48f7-9a4e-89a3947064d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b65ef8f2ca9446779a2f298b218ce1f9'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:37.246032 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8a2797d6-328e-48f7-9a4e-89a3947064d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b65ef8f2ca9446779a2f298b218ce1f9'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:37.247468 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8a2797d6-328e-48f7-9a4e-89a3947064d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b65ef8f2ca9446779a2f298b218ce1f9'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:37.273951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1079/2188] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:37 2022] DELETE /identity/v3/users/b65ef8f2ca9446779a2f298b218ce1f9 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:37.279296 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5aefa7-d5e9-4c46-8447-f6d13ceb75f5 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:37.281176 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5aefa7-d5e9-4c46-8447-f6d13ceb75f5 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:37.281910 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5aefa7-d5e9-4c46-8447-f6d13ceb75f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:37.314260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5b5aefa7-d5e9-4c46-8447-f6d13ceb75f5 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:37.317666 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5b5aefa7-d5e9-4c46-8447-f6d13ceb75f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:37.361423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:37.417824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1110/2189] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:07:37.429543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2797d6-328e-48f7-9a4e-89a3947064d7 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:37.431399 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8a2797d6-328e-48f7-9a4e-89a3947064d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:37.520127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2797d6-328e-48f7-9a4e-89a3947064d7 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:37.526946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e360433-bfb7-46b8-9097-194c7030e634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:37.528508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e360433-bfb7-46b8-9097-194c7030e634 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:37.529119 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e360433-bfb7-46b8-9097-194c7030e634 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:37.530260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e360433-bfb7-46b8-9097-194c7030e634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:37.546629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5aefa7-d5e9-4c46-8447-f6d13ceb75f5 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:37.552083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5b5aefa7-d5e9-4c46-8447-f6d13ceb75f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:37.560817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e360433-bfb7-46b8-9097-194c7030e634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:37.566049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e360433-bfb7-46b8-9097-194c7030e634 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:37.576562 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5aefa7-d5e9-4c46-8447-f6d13ceb75f5 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:37.594537 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1080/2190] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:37.597496 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ac6bc-67a0-4d24-a4a4-61e993095580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:37.598474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ac6bc-67a0-4d24-a4a4-61e993095580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:37.598757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ac6bc-67a0-4d24-a4a4-61e993095580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:37.599006 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ac6bc-67a0-4d24-a4a4-61e993095580 admin admin] PATH_INFO: `/v3/users/c5dc60b3eb544b428b43fe390b959d83` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:37.605574 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ac6bc-67a0-4d24-a4a4-61e993095580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5dc60b3eb544b428b43fe390b959d83)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:37.607603 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ac6bc-67a0-4d24-a4a4-61e993095580 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:37.723580 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-068ac6bc-67a0-4d24-a4a4-61e993095580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5dc60b3eb544b428b43fe390b959d83'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:37.744568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-068ac6bc-67a0-4d24-a4a4-61e993095580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5dc60b3eb544b428b43fe390b959d83'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:37.746665 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-068ac6bc-67a0-4d24-a4a4-61e993095580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5dc60b3eb544b428b43fe390b959d83'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:37.771901 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-068ac6bc-67a0-4d24-a4a4-61e993095580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5dc60b3eb544b428b43fe390b959d83'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:37.772482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-068ac6bc-67a0-4d24-a4a4-61e993095580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5dc60b3eb544b428b43fe390b959d83'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:37.806496 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1111/2191] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:37 2022] DELETE /identity/v3/users/c5dc60b3eb544b428b43fe390b959d83 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:37.811682 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e4d03-d44f-413e-b338-28389589a6da None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:37.812072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e4d03-d44f-413e-b338-28389589a6da None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:37.812521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e4d03-d44f-413e-b338-28389589a6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:37.849846 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-228e4d03-d44f-413e-b338-28389589a6da None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:37.851415 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-228e4d03-d44f-413e-b338-28389589a6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:37.951279 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1081/2192] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:07:37.965060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ac6bc-67a0-4d24-a4a4-61e993095580 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:37.968006 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-068ac6bc-67a0-4d24-a4a4-61e993095580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:38.031634 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e4d03-d44f-413e-b338-28389589a6da None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:38.033152 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-228e4d03-d44f-413e-b338-28389589a6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:38.062978 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e4d03-d44f-413e-b338-28389589a6da None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:38.084705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c025a-19cb-47e2-9ee3-32f58ad22441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:38.086175 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c025a-19cb-47e2-9ee3-32f58ad22441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:38.086381 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c025a-19cb-47e2-9ee3-32f58ad22441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:38.086665 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c025a-19cb-47e2-9ee3-32f58ad22441 admin admin] PATH_INFO: `/v3/projects/f0e7cd36281c4c54a0e7a9033bb22782` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:38.086782 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ac6bc-67a0-4d24-a4a4-61e993095580 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:38.091879 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c025a-19cb-47e2-9ee3-32f58ad22441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0e7cd36281c4c54a0e7a9033bb22782)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:38.096180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c025a-19cb-47e2-9ee3-32f58ad22441 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:38.103486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a53674-d787-490a-9356-45879a59b511 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:38.103816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a53674-d787-490a-9356-45879a59b511 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:38.104103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a53674-d787-490a-9356-45879a59b511 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:38.104400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a53674-d787-490a-9356-45879a59b511 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:38.115901 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a53674-d787-490a-9356-45879a59b511 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:38.117527 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a53674-d787-490a-9356-45879a59b511 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:38.129791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1112/2193] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:38.176487 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:38.194719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-078c025a-19cb-47e2-9ee3-32f58ad22441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0e7cd36281c4c54a0e7a9033bb22782', 'request_id': 'req-078c025a-19cb-47e2-9ee3-32f58ad22441'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:38.201536 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-078c025a-19cb-47e2-9ee3-32f58ad22441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0e7cd36281c4c54a0e7a9033bb22782', 'request_id': 'req-078c025a-19cb-47e2-9ee3-32f58ad22441'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:38.214399 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-078c025a-19cb-47e2-9ee3-32f58ad22441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0e7cd36281c4c54a0e7a9033bb22782', 'request_id': 'req-078c025a-19cb-47e2-9ee3-32f58ad22441'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:38.231273 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-078c025a-19cb-47e2-9ee3-32f58ad22441 admin admin] The token cache is being invalidate because project f0e7cd36281c4c54a0e7a9033bb22782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:38.231681 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-078c025a-19cb-47e2-9ee3-32f58ad22441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:38.232242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-078c025a-19cb-47e2-9ee3-32f58ad22441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:38.233863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1082/2194] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:38 2022] DELETE /identity/v3/projects/f0e7cd36281c4c54a0e7a9033bb22782 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:38.238156 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a53674-d787-490a-9356-45879a59b511 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:38.253923 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a53674-d787-490a-9356-45879a59b511 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:38.321783 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:38.323403 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c025a-19cb-47e2-9ee3-32f58ad22441 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:38.328908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-078c025a-19cb-47e2-9ee3-32f58ad22441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:38.331202 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b94f63-8387-4e15-89fa-927649d6991c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:38.332826 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b94f63-8387-4e15-89fa-927649d6991c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:38.333296 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b94f63-8387-4e15-89fa-927649d6991c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:38.333779 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b94f63-8387-4e15-89fa-927649d6991c admin admin] PATH_INFO: `/v3/users/bbc86f38ddde48eeb5e16dc2428e230d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:38.336152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b94f63-8387-4e15-89fa-927649d6991c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbc86f38ddde48eeb5e16dc2428e230d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:38.339966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b94f63-8387-4e15-89fa-927649d6991c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:38.344942 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c025a-19cb-47e2-9ee3-32f58ad22441 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:38.352493 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28338411-4fa5-4c5b-8fe1-bb94052b4f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:38.353775 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28338411-4fa5-4c5b-8fe1-bb94052b4f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:38.354123 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28338411-4fa5-4c5b-8fe1-bb94052b4f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:38.354889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28338411-4fa5-4c5b-8fe1-bb94052b4f95 admin admin] PATH_INFO: `/v3/users/a8c449e64d80404aa10a19a645da2cc1` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:38.356182 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28338411-4fa5-4c5b-8fe1-bb94052b4f95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8c449e64d80404aa10a19a645da2cc1)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:38.358798 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28338411-4fa5-4c5b-8fe1-bb94052b4f95 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:38.409160 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b1b94f63-8387-4e15-89fa-927649d6991c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbc86f38ddde48eeb5e16dc2428e230d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:38.416618 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b1b94f63-8387-4e15-89fa-927649d6991c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbc86f38ddde48eeb5e16dc2428e230d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:38.417332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b1b94f63-8387-4e15-89fa-927649d6991c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbc86f38ddde48eeb5e16dc2428e230d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:38.426336 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-28338411-4fa5-4c5b-8fe1-bb94052b4f95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8c449e64d80404aa10a19a645da2cc1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:38.427090 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-28338411-4fa5-4c5b-8fe1-bb94052b4f95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8c449e64d80404aa10a19a645da2cc1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:38.428788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b1b94f63-8387-4e15-89fa-927649d6991c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbc86f38ddde48eeb5e16dc2428e230d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:38.430811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b1b94f63-8387-4e15-89fa-927649d6991c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbc86f38ddde48eeb5e16dc2428e230d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:38.437406 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-28338411-4fa5-4c5b-8fe1-bb94052b4f95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8c449e64d80404aa10a19a645da2cc1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:38.447171 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-28338411-4fa5-4c5b-8fe1-bb94052b4f95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8c449e64d80404aa10a19a645da2cc1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:38.447980 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-28338411-4fa5-4c5b-8fe1-bb94052b4f95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8c449e64d80404aa10a19a645da2cc1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:38.457645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1113/2195] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:38 2022] DELETE /identity/v3/users/bbc86f38ddde48eeb5e16dc2428e230d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:38.474517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1083/2196] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:38 2022] DELETE /identity/v3/users/a8c449e64d80404aa10a19a645da2cc1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:38.479573 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d41db-6d6b-400f-abeb-12009d3c2245 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:38.479857 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d41db-6d6b-400f-abeb-12009d3c2245 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:38.480136 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d41db-6d6b-400f-abeb-12009d3c2245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:38.500200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-471d41db-6d6b-400f-abeb-12009d3c2245 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:38.501670 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-471d41db-6d6b-400f-abeb-12009d3c2245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:38.592771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1114/2197] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:38.618239 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28338411-4fa5-4c5b-8fe1-bb94052b4f95 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:38.621658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-28338411-4fa5-4c5b-8fe1-bb94052b4f95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:38.750141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28338411-4fa5-4c5b-8fe1-bb94052b4f95 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:38.755108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd388d-eaab-46fb-bce5-69a84efc3d24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:38.756116 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd388d-eaab-46fb-bce5-69a84efc3d24 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:38.756602 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd388d-eaab-46fb-bce5-69a84efc3d24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:38.757017 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd388d-eaab-46fb-bce5-69a84efc3d24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:38.770257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd388d-eaab-46fb-bce5-69a84efc3d24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:38.772986 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd388d-eaab-46fb-bce5-69a84efc3d24 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:38.794404 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1084/2198] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:38.906763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d41db-6d6b-400f-abeb-12009d3c2245 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:38.910677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-471d41db-6d6b-400f-abeb-12009d3c2245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:38.934876 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d41db-6d6b-400f-abeb-12009d3c2245 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:38.981123 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4311c7b-79e7-49ad-af51-a79e41472f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:38.982026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4311c7b-79e7-49ad-af51-a79e41472f8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:38.982297 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4311c7b-79e7-49ad-af51-a79e41472f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:38.982574 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4311c7b-79e7-49ad-af51-a79e41472f8f admin admin] PATH_INFO: `/v3/projects/72d881a31a804abc9385c9d1cb94dd16` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:38.988467 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4311c7b-79e7-49ad-af51-a79e41472f8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=72d881a31a804abc9385c9d1cb94dd16)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:38.995578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4311c7b-79e7-49ad-af51-a79e41472f8f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:38.997843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd388d-eaab-46fb-bce5-69a84efc3d24 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:38.999815 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e3fd388d-eaab-46fb-bce5-69a84efc3d24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:39.021049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd388d-eaab-46fb-bce5-69a84efc3d24 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:39.075384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd5219-95cd-4ad1-ad80-f879f999c20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:39.096717 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd5219-95cd-4ad1-ad80-f879f999c20d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:39.097222 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd5219-95cd-4ad1-ad80-f879f999c20d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:39.097742 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd5219-95cd-4ad1-ad80-f879f999c20d admin admin] PATH_INFO: `/v3/projects/1e5e5870c28b4400ade619e753e67f7b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:39.102789 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a4311c7b-79e7-49ad-af51-a79e41472f8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72d881a31a804abc9385c9d1cb94dd16', 'request_id': 'req-a4311c7b-79e7-49ad-af51-a79e41472f8f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.108867 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd5219-95cd-4ad1-ad80-f879f999c20d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e5e5870c28b4400ade619e753e67f7b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:39.111772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd5219-95cd-4ad1-ad80-f879f999c20d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:39.116588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a4311c7b-79e7-49ad-af51-a79e41472f8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72d881a31a804abc9385c9d1cb94dd16', 'request_id': 'req-a4311c7b-79e7-49ad-af51-a79e41472f8f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.137439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a4311c7b-79e7-49ad-af51-a79e41472f8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72d881a31a804abc9385c9d1cb94dd16', 'request_id': 'req-a4311c7b-79e7-49ad-af51-a79e41472f8f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.142939 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:39.170792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a4311c7b-79e7-49ad-af51-a79e41472f8f admin admin] The token cache is being invalidate because project 72d881a31a804abc9385c9d1cb94dd16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:39.171090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a4311c7b-79e7-49ad-af51-a79e41472f8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.171964 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a4311c7b-79e7-49ad-af51-a79e41472f8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.180501 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1115/2199] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:38 2022] DELETE /identity/v3/projects/72d881a31a804abc9385c9d1cb94dd16 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:39.185514 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd690af8-e6e1-4b6c-8cc0-d3338dcc8664 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:39.185829 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd690af8-e6e1-4b6c-8cc0-d3338dcc8664 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:39.186123 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd690af8-e6e1-4b6c-8cc0-d3338dcc8664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:39.229158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-fd690af8-e6e1-4b6c-8cc0-d3338dcc8664 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:39.231744 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-fd690af8-e6e1-4b6c-8cc0-d3338dcc8664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:39.261959 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:39.290372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-44fd5219-95cd-4ad1-ad80-f879f999c20d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e5e5870c28b4400ade619e753e67f7b', 'request_id': 'req-44fd5219-95cd-4ad1-ad80-f879f999c20d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.299974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-44fd5219-95cd-4ad1-ad80-f879f999c20d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e5e5870c28b4400ade619e753e67f7b', 'request_id': 'req-44fd5219-95cd-4ad1-ad80-f879f999c20d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.313917 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-44fd5219-95cd-4ad1-ad80-f879f999c20d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e5e5870c28b4400ade619e753e67f7b', 'request_id': 'req-44fd5219-95cd-4ad1-ad80-f879f999c20d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.331838 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-44fd5219-95cd-4ad1-ad80-f879f999c20d admin admin] The token cache is being invalidate because project 1e5e5870c28b4400ade619e753e67f7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:39.332218 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-44fd5219-95cd-4ad1-ad80-f879f999c20d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.335194 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-44fd5219-95cd-4ad1-ad80-f879f999c20d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.337789 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1085/2200] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:38 2022] DELETE /identity/v3/projects/1e5e5870c28b4400ade619e753e67f7b => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:39.341738 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd5219-95cd-4ad1-ad80-f879f999c20d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:39.343546 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-44fd5219-95cd-4ad1-ad80-f879f999c20d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:39.398680 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1116/2201] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:39.406695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8af72d-2bfa-4ece-8374-f01e9f573f04 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:39.411829 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8af72d-2bfa-4ece-8374-f01e9f573f04 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:39.412996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8af72d-2bfa-4ece-8374-f01e9f573f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:39.449681 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5d8af72d-2bfa-4ece-8374-f01e9f573f04 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:39.449681 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5d8af72d-2bfa-4ece-8374-f01e9f573f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:39.481424 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd5219-95cd-4ad1-ad80-f879f999c20d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:39.487524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4beec6-76bd-4b41-9aa4-76000484d456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:39.488938 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4beec6-76bd-4b41-9aa4-76000484d456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:39.489648 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4beec6-76bd-4b41-9aa4-76000484d456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:39.490387 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4beec6-76bd-4b41-9aa4-76000484d456 admin admin] PATH_INFO: `/v3/users/2eee7044df9b45d6bc32f716d1df5c83` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:39.492559 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4beec6-76bd-4b41-9aa4-76000484d456 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eee7044df9b45d6bc32f716d1df5c83)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:39.495315 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4beec6-76bd-4b41-9aa4-76000484d456 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:39.610611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cd4beec6-76bd-4b41-9aa4-76000484d456 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eee7044df9b45d6bc32f716d1df5c83'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.611586 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cd4beec6-76bd-4b41-9aa4-76000484d456 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eee7044df9b45d6bc32f716d1df5c83'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.630164 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cd4beec6-76bd-4b41-9aa4-76000484d456 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eee7044df9b45d6bc32f716d1df5c83'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.648020 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cd4beec6-76bd-4b41-9aa4-76000484d456 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eee7044df9b45d6bc32f716d1df5c83'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.649600 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-cd4beec6-76bd-4b41-9aa4-76000484d456 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eee7044df9b45d6bc32f716d1df5c83'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.672166 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1117/2202] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:39.674231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8af72d-2bfa-4ece-8374-f01e9f573f04 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:39.676700 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5d8af72d-2bfa-4ece-8374-f01e9f573f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:39.693609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1086/2203] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:39 2022] DELETE /identity/v3/users/2eee7044df9b45d6bc32f716d1df5c83 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:39.694615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4beec6-76bd-4b41-9aa4-76000484d456 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:39.704277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8af72d-2bfa-4ece-8374-f01e9f573f04 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:39.778044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:39.802202 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88cae4-f4b4-441d-ab1a-28fab795f806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:39.803821 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88cae4-f4b4-441d-ab1a-28fab795f806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:39.804358 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88cae4-f4b4-441d-ab1a-28fab795f806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:39.804670 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88cae4-f4b4-441d-ab1a-28fab795f806 admin admin] PATH_INFO: `/v3/projects/a60af8d79166435d9147690a742b5709` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:39.806905 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88cae4-f4b4-441d-ab1a-28fab795f806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a60af8d79166435d9147690a742b5709)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:39.809107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88cae4-f4b4-441d-ab1a-28fab795f806 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:39.810193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4beec6-76bd-4b41-9aa4-76000484d456 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:39.815933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a665f89-cd01-405c-92eb-ccc713de0467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:39.817477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a665f89-cd01-405c-92eb-ccc713de0467 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:39.817874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a665f89-cd01-405c-92eb-ccc713de0467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:39.818295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a665f89-cd01-405c-92eb-ccc713de0467 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:39.819624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a665f89-cd01-405c-92eb-ccc713de0467 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:07:39.820065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a665f89-cd01-405c-92eb-ccc713de0467 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:39.822324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a665f89-cd01-405c-92eb-ccc713de0467 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:39.862362 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1087/2204] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:07:39.867807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a665f89-cd01-405c-92eb-ccc713de0467 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:39.884255 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:39.895173 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a665f89-cd01-405c-92eb-ccc713de0467 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:39.912667 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2c88cae4-f4b4-441d-ab1a-28fab795f806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a60af8d79166435d9147690a742b5709', 'request_id': 'req-2c88cae4-f4b4-441d-ab1a-28fab795f806'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.927173 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2c88cae4-f4b4-441d-ab1a-28fab795f806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a60af8d79166435d9147690a742b5709', 'request_id': 'req-2c88cae4-f4b4-441d-ab1a-28fab795f806'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.938360 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2c88cae4-f4b4-441d-ab1a-28fab795f806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a60af8d79166435d9147690a742b5709', 'request_id': 'req-2c88cae4-f4b4-441d-ab1a-28fab795f806'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.963486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2c88cae4-f4b4-441d-ab1a-28fab795f806 admin admin] The token cache is being invalidate because project a60af8d79166435d9147690a742b5709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:39.964322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2c88cae4-f4b4-441d-ab1a-28fab795f806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.966244 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2c88cae4-f4b4-441d-ab1a-28fab795f806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:39.968493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1118/2205] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:39 2022] DELETE /identity/v3/projects/a60af8d79166435d9147690a742b5709 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:39.975105 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f038e94-652a-47d6-8538-e94cf48e88b7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:39.975925 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f038e94-652a-47d6-8538-e94cf48e88b7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:39.976137 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f038e94-652a-47d6-8538-e94cf48e88b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:39.988717 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bec519-d6e3-42aa-a6e3-cf79da3ab59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:39.989963 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bec519-d6e3-42aa-a6e3-cf79da3ab59d admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:39.990499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bec519-d6e3-42aa-a6e3-cf79da3ab59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:39.991099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bec519-d6e3-42aa-a6e3-cf79da3ab59d admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:39.992012 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bec519-d6e3-42aa-a6e3-cf79da3ab59d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:39.994417 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bec519-d6e3-42aa-a6e3-cf79da3ab59d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:40.009607 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1f038e94-652a-47d6-8538-e94cf48e88b7 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:40.011077 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1f038e94-652a-47d6-8538-e94cf48e88b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:40.031103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1119/2206] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:40.036649 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d990b3-446a-4028-8f79-fb040956d3d4 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:40.037941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d990b3-446a-4028-8f79-fb040956d3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:40.038526 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d990b3-446a-4028-8f79-fb040956d3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:40.076189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-d5d990b3-446a-4028-8f79-fb040956d3d4 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:40.080557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d5d990b3-446a-4028-8f79-fb040956d3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:40.087215 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1088/2207] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:39 2022] POST /identity/v3/projects => generated 373 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:40.091040 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bec519-d6e3-42aa-a6e3-cf79da3ab59d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:40.095872 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-88bec519-d6e3-42aa-a6e3-cf79da3ab59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:40.105557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1120/2208] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:40.107874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d990b3-446a-4028-8f79-fb040956d3d4 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:40.138628 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d990b3-446a-4028-8f79-fb040956d3d4 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:40.249108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bec519-d6e3-42aa-a6e3-cf79da3ab59d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:40.262313 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bfe18-ba10-45c5-b96c-cccbd9a665d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:40.263639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bfe18-ba10-45c5-b96c-cccbd9a665d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:40.264045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bfe18-ba10-45c5-b96c-cccbd9a665d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:40.264328 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bfe18-ba10-45c5-b96c-cccbd9a665d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:40.266946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bfe18-ba10-45c5-b96c-cccbd9a665d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:40.271920 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bfe18-ba10-45c5-b96c-cccbd9a665d7 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:40.308468 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1089/2209] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:40 2022] POST /identity/v3/users => generated 287 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:40.313827 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bfe18-ba10-45c5-b96c-cccbd9a665d7 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:40.316505 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a0b51-5eae-4e3c-8edb-84793b293a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:40.317224 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a0b51-5eae-4e3c-8edb-84793b293a81 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:40.317504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a0b51-5eae-4e3c-8edb-84793b293a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:40.317781 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a0b51-5eae-4e3c-8edb-84793b293a81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:40.320002 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a0b51-5eae-4e3c-8edb-84793b293a81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:07:40.321707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a0b51-5eae-4e3c-8edb-84793b293a81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:40.326496 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bfe18-ba10-45c5-b96c-cccbd9a665d7 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:40.327746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a0b51-5eae-4e3c-8edb-84793b293a81 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:40.333889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef54ba-9163-4b7b-a48e-16dc8e290c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:40.334828 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef54ba-9163-4b7b-a48e-16dc8e290c4f admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:40.335108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef54ba-9163-4b7b-a48e-16dc8e290c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:40.336247 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef54ba-9163-4b7b-a48e-16dc8e290c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:40.337068 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cef54ba-9163-4b7b-a48e-16dc8e290c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:40.338777 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cef54ba-9163-4b7b-a48e-16dc8e290c4f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:40.362271 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1090/2210] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:40.365555 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef54ba-9163-4b7b-a48e-16dc8e290c4f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:40.366651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1121/2211] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:07:40.371400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a0b51-5eae-4e3c-8edb-84793b293a81 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:40.383411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef54ba-9163-4b7b-a48e-16dc8e290c4f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:40.400674 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522697f-714f-40b3-8288-63c098950817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:40.402473 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522697f-714f-40b3-8288-63c098950817 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:40.402593 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522697f-714f-40b3-8288-63c098950817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:40.402894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522697f-714f-40b3-8288-63c098950817 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:40.405671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522697f-714f-40b3-8288-63c098950817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:40.411193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522697f-714f-40b3-8288-63c098950817 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:40.415671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a0b51-5eae-4e3c-8edb-84793b293a81 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:40.436801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123952ee-bc42-4cca-b6f0-914d158fe68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:40.437907 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123952ee-bc42-4cca-b6f0-914d158fe68d admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:40.439972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123952ee-bc42-4cca-b6f0-914d158fe68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:40.440602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123952ee-bc42-4cca-b6f0-914d158fe68d admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:40.441589 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123952ee-bc42-4cca-b6f0-914d158fe68d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:40.446223 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123952ee-bc42-4cca-b6f0-914d158fe68d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:40.451352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1091/2212] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:07:40.457907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522697f-714f-40b3-8288-63c098950817 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:40.495314 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522697f-714f-40b3-8288-63c098950817 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:40.511671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660211d-658b-4331-9c0b-ba0b2217e587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:40.512812 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660211d-658b-4331-9c0b-ba0b2217e587 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:40.513094 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660211d-658b-4331-9c0b-ba0b2217e587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:40.513376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660211d-658b-4331-9c0b-ba0b2217e587 admin admin] PATH_INFO: `/v3/projects/2245fe1962f2499db7c429815fd4d8ef/users/ba1c6c6fa9e44cf080f103bf98b04e5f/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:40.513759 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1122/2213] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:40 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:40.520301 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123952ee-bc42-4cca-b6f0-914d158fe68d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:40.560749 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660211d-658b-4331-9c0b-ba0b2217e587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2245fe1962f2499db7c429815fd4d8ef, user_id=ba1c6c6fa9e44cf080f103bf98b04e5f, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:40.567072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660211d-658b-4331-9c0b-ba0b2217e587 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:40.597697 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1092/2214] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:40 2022] PUT /identity/v3/projects/2245fe1962f2499db7c429815fd4d8ef/users/ba1c6c6fa9e44cf080f103bf98b04e5f/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:40.612624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962acdf-f3e7-4b9d-8fb8-b0ca3e61786e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:40.614689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962acdf-f3e7-4b9d-8fb8-b0ca3e61786e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:40.614689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962acdf-f3e7-4b9d-8fb8-b0ca3e61786e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:40.632963 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:40.674089 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-1962acdf-f3e7-4b9d-8fb8-b0ca3e61786e None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:40.674089 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1962acdf-f3e7-4b9d-8fb8-b0ca3e61786e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:40.685982 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123952ee-bc42-4cca-b6f0-914d158fe68d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:40.754791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:40.859436 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065ad22-9d32-4ec4-83b4-b4a7b99003f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:40.868399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065ad22-9d32-4ec4-83b4-b4a7b99003f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:40.868748 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065ad22-9d32-4ec4-83b4-b4a7b99003f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:40.869185 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065ad22-9d32-4ec4-83b4-b4a7b99003f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:40.870331 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065ad22-9d32-4ec4-83b4-b4a7b99003f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:40.875554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3065ad22-9d32-4ec4-83b4-b4a7b99003f1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:40.879371 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1093/2215] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:40.882362 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962acdf-f3e7-4b9d-8fb8-b0ca3e61786e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:40.885050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1962acdf-f3e7-4b9d-8fb8-b0ca3e61786e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:40.912976 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1123/2216] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:40 2022] POST /identity/v3/users => generated 286 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:40.913926 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065ad22-9d32-4ec4-83b4-b4a7b99003f1 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:40.916801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962acdf-f3e7-4b9d-8fb8-b0ca3e61786e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:40.920279 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3065ad22-9d32-4ec4-83b4-b4a7b99003f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:40.931688 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d11f3-8d80-4223-8567-300e811b919c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:40.932616 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d11f3-8d80-4223-8567-300e811b919c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:40.933073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d11f3-8d80-4223-8567-300e811b919c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:40.933361 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d11f3-8d80-4223-8567-300e811b919c admin admin] PATH_INFO: `/v3/projects/74f834722df641f0ba6a44f86d4e1db4` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:40.935240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d11f3-8d80-4223-8567-300e811b919c admin admin] RBAC: Authorizing `identity:delete_project(project_id=74f834722df641f0ba6a44f86d4e1db4)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:40.937237 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d11f3-8d80-4223-8567-300e811b919c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:41.017476 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d82d11f3-8d80-4223-8567-300e811b919c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74f834722df641f0ba6a44f86d4e1db4', 'request_id': 'req-d82d11f3-8d80-4223-8567-300e811b919c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:41.022030 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065ad22-9d32-4ec4-83b4-b4a7b99003f1 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:41.023474 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d82d11f3-8d80-4223-8567-300e811b919c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74f834722df641f0ba6a44f86d4e1db4', 'request_id': 'req-d82d11f3-8d80-4223-8567-300e811b919c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:41.048522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d82d11f3-8d80-4223-8567-300e811b919c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74f834722df641f0ba6a44f86d4e1db4', 'request_id': 'req-d82d11f3-8d80-4223-8567-300e811b919c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:41.078470 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d82d11f3-8d80-4223-8567-300e811b919c admin admin] The token cache is being invalidate because project 74f834722df641f0ba6a44f86d4e1db4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:41.078470 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d82d11f3-8d80-4223-8567-300e811b919c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:41.099339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d82d11f3-8d80-4223-8567-300e811b919c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:41.106113 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1094/2217] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:40 2022] DELETE /identity/v3/projects/74f834722df641f0ba6a44f86d4e1db4 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:41.106113 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d11f3-8d80-4223-8567-300e811b919c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:41.106113 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d82d11f3-8d80-4223-8567-300e811b919c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:41.153428 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa808c8d-9cc8-48f8-92b6-1081781616e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:41.156017 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa808c8d-9cc8-48f8-92b6-1081781616e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:41.156321 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa808c8d-9cc8-48f8-92b6-1081781616e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:41.156638 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa808c8d-9cc8-48f8-92b6-1081781616e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:41.162379 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-aa808c8d-9cc8-48f8-92b6-1081781616e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:41.235148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d11f3-8d80-4223-8567-300e811b919c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:41.266844 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781ad95-33b8-4a28-a81b-796ae13ec432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:41.268550 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781ad95-33b8-4a28-a81b-796ae13ec432 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:41.268837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781ad95-33b8-4a28-a81b-796ae13ec432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:41.269112 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781ad95-33b8-4a28-a81b-796ae13ec432 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:41.269860 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781ad95-33b8-4a28-a81b-796ae13ec432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:41.286905 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781ad95-33b8-4a28-a81b-796ae13ec432 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:41.308590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1095/2218] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:41.309981 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a5c9c-4a60-4433-9855-4186f66b127e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:41.310195 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa808c8d-9cc8-48f8-92b6-1081781616e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:41.310315 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a5c9c-4a60-4433-9855-4186f66b127e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:41.310529 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a5c9c-4a60-4433-9855-4186f66b127e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:41.311746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa808c8d-9cc8-48f8-92b6-1081781616e9 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:41.334563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1124/2219] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:41.335637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa808c8d-9cc8-48f8-92b6-1081781616e9 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:41.338152 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-976a5c9c-4a60-4433-9855-4186f66b127e None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:41.339556 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-976a5c9c-4a60-4433-9855-4186f66b127e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:41.359501 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1096/2220] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:41.362075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35ccde-8864-4be2-b890-4576138c8bde None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:41.362295 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa808c8d-9cc8-48f8-92b6-1081781616e9 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:41.363066 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35ccde-8864-4be2-b890-4576138c8bde None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:41.363558 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35ccde-8864-4be2-b890-4576138c8bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:41.372088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001b671-6bb9-45ab-8676-68c25742247b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:41.372801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001b671-6bb9-45ab-8676-68c25742247b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:41.373182 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001b671-6bb9-45ab-8676-68c25742247b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:41.373354 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001b671-6bb9-45ab-8676-68c25742247b admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:41.374077 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001b671-6bb9-45ab-8676-68c25742247b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:41.375686 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001b671-6bb9-45ab-8676-68c25742247b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:41.388770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-3a35ccde-8864-4be2-b890-4576138c8bde None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:41.390703 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3a35ccde-8864-4be2-b890-4576138c8bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:41.400695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1125/2221] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:41.401411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1097/2222] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:41.405478 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35ccde-8864-4be2-b890-4576138c8bde None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:41.405669 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001b671-6bb9-45ab-8676-68c25742247b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:41.421105 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35ccde-8864-4be2-b890-4576138c8bde None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:41.423045 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001b671-6bb9-45ab-8676-68c25742247b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:41.437707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd96d2-db29-4f2e-89ed-95031c54278a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:41.438641 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd96d2-db29-4f2e-89ed-95031c54278a admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:41.438926 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd96d2-db29-4f2e-89ed-95031c54278a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:41.439202 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd96d2-db29-4f2e-89ed-95031c54278a admin admin] PATH_INFO: `/v3/projects/9b1a552fc6cc43bb80f8a5f951bfcf80/users/5bd682806c864805a3b410411839ca99/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:41.452022 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19dd05e-6506-4f74-b7b5-3f47980222d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:41.453548 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dd05e-6506-4f74-b7b5-3f47980222d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:41.454046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dd05e-6506-4f74-b7b5-3f47980222d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:41.454660 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dd05e-6506-4f74-b7b5-3f47980222d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:41.455959 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19dd05e-6506-4f74-b7b5-3f47980222d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:07:41.456491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19dd05e-6506-4f74-b7b5-3f47980222d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:41.458906 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19dd05e-6506-4f74-b7b5-3f47980222d3 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:41.464673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:41.472967 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:41.474696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd96d2-db29-4f2e-89ed-95031c54278a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b1a552fc6cc43bb80f8a5f951bfcf80, user_id=5bd682806c864805a3b410411839ca99, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:41.477869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd96d2-db29-4f2e-89ed-95031c54278a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:41.494569 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1098/2223] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:07:41.496211 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19dd05e-6506-4f74-b7b5-3f47980222d3 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:41.513339 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1126/2224] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:41 2022] PUT /identity/v3/projects/9b1a552fc6cc43bb80f8a5f951bfcf80/users/5bd682806c864805a3b410411839ca99/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:41.515505 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19dd05e-6506-4f74-b7b5-3f47980222d3 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:41.516561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55fba7-1e82-4136-ab85-6554bbdeeb73 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:41.517065 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55fba7-1e82-4136-ab85-6554bbdeeb73 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:41.517745 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55fba7-1e82-4136-ab85-6554bbdeeb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:41.540565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-cc55fba7-1e82-4136-ab85-6554bbdeeb73 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:41.542167 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-cc55fba7-1e82-4136-ab85-6554bbdeeb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:41.667553 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c397852-2ed2-482c-a73c-24955e3f9ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:41.669582 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c397852-2ed2-482c-a73c-24955e3f9ace admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:41.670147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c397852-2ed2-482c-a73c-24955e3f9ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:41.670675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c397852-2ed2-482c-a73c-24955e3f9ace admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:41.672066 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c397852-2ed2-482c-a73c-24955e3f9ace admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:41.675347 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c397852-2ed2-482c-a73c-24955e3f9ace admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:41.691288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1127/2225] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:41.710521 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55fba7-1e82-4136-ab85-6554bbdeeb73 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:41.712808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-cc55fba7-1e82-4136-ab85-6554bbdeeb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:41.746313 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1099/2226] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:41 2022] POST /identity/v3/projects => generated 373 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:41.747445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c397852-2ed2-482c-a73c-24955e3f9ace admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:41.877376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55fba7-1e82-4136-ab85-6554bbdeeb73 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:41.938341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c397852-2ed2-482c-a73c-24955e3f9ace admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:41.954453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f020aaa-8e31-43cf-ae4d-ec527daf5a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:41.958374 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f020aaa-8e31-43cf-ae4d-ec527daf5a58 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:41.958924 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f020aaa-8e31-43cf-ae4d-ec527daf5a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:41.959209 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f020aaa-8e31-43cf-ae4d-ec527daf5a58 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:41.961517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f020aaa-8e31-43cf-ae4d-ec527daf5a58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:41.969223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f020aaa-8e31-43cf-ae4d-ec527daf5a58 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:41.990767 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07170dab-e6c3-443a-bd25-e88cf05627c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:41.992018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07170dab-e6c3-443a-bd25-e88cf05627c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:41.992296 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07170dab-e6c3-443a-bd25-e88cf05627c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:41.994196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07170dab-e6c3-443a-bd25-e88cf05627c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:42.028622 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07170dab-e6c3-443a-bd25-e88cf05627c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:42.033430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1100/2227] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:41 2022] POST /identity/v3/users => generated 287 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:42.033604 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07170dab-e6c3-443a-bd25-e88cf05627c2 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:42.035026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f020aaa-8e31-43cf-ae4d-ec527daf5a58 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:42.057932 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f020aaa-8e31-43cf-ae4d-ec527daf5a58 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:42.059348 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1128/2228] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:42.064016 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22a7c0-294d-4344-9bb9-c81000812614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:42.064749 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22a7c0-294d-4344-9bb9-c81000812614 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:42.065033 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22a7c0-294d-4344-9bb9-c81000812614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:42.065387 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22a7c0-294d-4344-9bb9-c81000812614 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:42.066047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22a7c0-294d-4344-9bb9-c81000812614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:42.068875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22a7c0-294d-4344-9bb9-c81000812614 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:42.083866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1101/2229] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:42.085709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07170dab-e6c3-443a-bd25-e88cf05627c2 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:42.097923 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07170dab-e6c3-443a-bd25-e88cf05627c2 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:42.103642 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24dd852-c52f-4dc3-b23c-78f6257f2b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:42.104598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24dd852-c52f-4dc3-b23c-78f6257f2b2e admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:42.104885 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24dd852-c52f-4dc3-b23c-78f6257f2b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:42.105367 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24dd852-c52f-4dc3-b23c-78f6257f2b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:42.106559 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24dd852-c52f-4dc3-b23c-78f6257f2b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:42.108965 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24dd852-c52f-4dc3-b23c-78f6257f2b2e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:42.126590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1129/2230] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:42.127934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22a7c0-294d-4344-9bb9-c81000812614 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:42.138807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22a7c0-294d-4344-9bb9-c81000812614 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:42.154362 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001694e-0999-4b28-9cc0-439d6bf57143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:42.155405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001694e-0999-4b28-9cc0-439d6bf57143 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:42.155691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001694e-0999-4b28-9cc0-439d6bf57143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:42.155970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001694e-0999-4b28-9cc0-439d6bf57143 admin admin] PATH_INFO: `/v3/projects/6d67bef49576408e96d7fdd3853aa7b0/users/9f9b5f17671940df9572e9b58af9b88f/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:42.185058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3001694e-0999-4b28-9cc0-439d6bf57143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d67bef49576408e96d7fdd3853aa7b0, user_id=9f9b5f17671940df9572e9b58af9b88f, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:42.187750 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3001694e-0999-4b28-9cc0-439d6bf57143 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:42.202892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24dd852-c52f-4dc3-b23c-78f6257f2b2e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:42.204692 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b24dd852-c52f-4dc3-b23c-78f6257f2b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:42.215634 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1102/2231] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:42 2022] PUT /identity/v3/projects/6d67bef49576408e96d7fdd3853aa7b0/users/9f9b5f17671940df9572e9b58af9b88f/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:42.230407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ab561-31eb-439e-9f4a-1cf7cbd251d6 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:42.230407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ab561-31eb-439e-9f4a-1cf7cbd251d6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:42.233194 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ab561-31eb-439e-9f4a-1cf7cbd251d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:42.285480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e59ab561-31eb-439e-9f4a-1cf7cbd251d6 None None] MFA Rules not processed for user `9f9b5f17671940df9572e9b58af9b88f`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:42.287019 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e59ab561-31eb-439e-9f4a-1cf7cbd251d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:42.300819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:42.330801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:42.335031 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24dd852-c52f-4dc3-b23c-78f6257f2b2e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:42.418778 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1103/2232] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:07:42.435295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ab561-31eb-439e-9f4a-1cf7cbd251d6 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:42.438687 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e59ab561-31eb-439e-9f4a-1cf7cbd251d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:42.476987 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8929a-6ac8-456d-aead-cfd530519108 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:42.478389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8929a-6ac8-456d-aead-cfd530519108 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:42.478918 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8929a-6ac8-456d-aead-cfd530519108 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:42.481072 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8929a-6ac8-456d-aead-cfd530519108 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:42.485352 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1ab8929a-6ac8-456d-aead-cfd530519108 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:42.604588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab8929a-6ac8-456d-aead-cfd530519108 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:42.606593 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab8929a-6ac8-456d-aead-cfd530519108 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:42.619058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1130/2233] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:07:42.627741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ab561-31eb-439e-9f4a-1cf7cbd251d6 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:42.632968 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87eb01-bfbe-4df6-998c-32696e4ea79e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:42.633905 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87eb01-bfbe-4df6-998c-32696e4ea79e service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:42.634420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87eb01-bfbe-4df6-998c-32696e4ea79e service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:42.634843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87eb01-bfbe-4df6-998c-32696e4ea79e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:42.671296 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87eb01-bfbe-4df6-998c-32696e4ea79e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:42.675114 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87eb01-bfbe-4df6-998c-32696e4ea79e service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:42.720713 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1104/2234] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:42.754571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8929a-6ac8-456d-aead-cfd530519108 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:42.942902 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8929a-6ac8-456d-aead-cfd530519108 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:42.943333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2fbe3-2a7e-4094-bb29-651687ce24d5 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:42.943705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2fbe3-2a7e-4094-bb29-651687ce24d5 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:42.944354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce2fbe3-2a7e-4094-bb29-651687ce24d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:42.953308 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d6b77-657b-4beb-a5b3-2382bc187424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:42.954348 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d6b77-657b-4beb-a5b3-2382bc187424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:42.954573 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d6b77-657b-4beb-a5b3-2382bc187424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:42.954794 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d6b77-657b-4beb-a5b3-2382bc187424 admin admin] PATH_INFO: `/v3/users/9f9b5f17671940df9572e9b58af9b88f` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:42.956023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d6b77-657b-4beb-a5b3-2382bc187424 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f9b5f17671940df9572e9b58af9b88f)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:42.958013 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d6b77-657b-4beb-a5b3-2382bc187424 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:42.999774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-4ce2fbe3-2a7e-4094-bb29-651687ce24d5 None None] MFA Rules not processed for user `744eb9d4e7084b7e9cae36d5de67f0f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:43.001314 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4ce2fbe3-2a7e-4094-bb29-651687ce24d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:43.056930 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-697d6b77-657b-4beb-a5b3-2382bc187424 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f9b5f17671940df9572e9b58af9b88f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:43.060215 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:43.074975 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-697d6b77-657b-4beb-a5b3-2382bc187424 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f9b5f17671940df9572e9b58af9b88f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:43.075806 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-697d6b77-657b-4beb-a5b3-2382bc187424 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f9b5f17671940df9572e9b58af9b88f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:43.093023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-697d6b77-657b-4beb-a5b3-2382bc187424 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f9b5f17671940df9572e9b58af9b88f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:43.093826 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-697d6b77-657b-4beb-a5b3-2382bc187424 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f9b5f17671940df9572e9b58af9b88f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:43.162402 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1131/2235] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:42 2022] DELETE /identity/v3/users/9f9b5f17671940df9572e9b58af9b88f => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:43.166802 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca15e4-d1bb-4e1e-9abc-8a70531df248 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:43.167073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca15e4-d1bb-4e1e-9abc-8a70531df248 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:43.167363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca15e4-d1bb-4e1e-9abc-8a70531df248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:43.196368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1105/2236] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:07:43.211560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3bca15e4-d1bb-4e1e-9abc-8a70531df248 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:43.213079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3bca15e4-d1bb-4e1e-9abc-8a70531df248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:43.215809 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2fbe3-2a7e-4094-bb29-651687ce24d5 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:43.217242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4ce2fbe3-2a7e-4094-bb29-651687ce24d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:43.328735 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:43.360299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1132/2237] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:43.370013 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce2fbe3-2a7e-4094-bb29-651687ce24d5 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:43.371073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca15e4-d1bb-4e1e-9abc-8a70531df248 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:43.373682 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3bca15e4-d1bb-4e1e-9abc-8a70531df248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:43.468969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5997e-2555-4669-8299-cccf6d045999 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:43.470445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5997e-2555-4669-8299-cccf6d045999 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:43.470445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5997e-2555-4669-8299-cccf6d045999 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:43.470893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5997e-2555-4669-8299-cccf6d045999 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:43.502404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca15e4-d1bb-4e1e-9abc-8a70531df248 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:43.508054 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b9b0a-57f1-4926-9775-ffa80e17a506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:43.508758 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b9b0a-57f1-4926-9775-ffa80e17a506 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:43.509091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b9b0a-57f1-4926-9775-ffa80e17a506 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:43.509370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b9b0a-57f1-4926-9775-ffa80e17a506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:43.535611 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b9b0a-57f1-4926-9775-ffa80e17a506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:43.537911 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b9b0a-57f1-4926-9775-ffa80e17a506 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:43.569242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1133/2238] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:43.671386 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5997e-2555-4669-8299-cccf6d045999 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:43.673594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5997e-2555-4669-8299-cccf6d045999 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:43.698045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1106/2239] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:43.807976 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30a036-3596-460c-a4ea-051e6817e4c7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:43.808264 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30a036-3596-460c-a4ea-051e6817e4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:43.808600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30a036-3596-460c-a4ea-051e6817e4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:43.874100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-6f30a036-3596-460c-a4ea-051e6817e4c7 None None] MFA Rules not processed for user `ba1c6c6fa9e44cf080f103bf98b04e5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:43.876845 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6f30a036-3596-460c-a4ea-051e6817e4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:44.070069 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:44.079892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1134/2240] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:07:44.080684 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5997e-2555-4669-8299-cccf6d045999 service nova] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:44.082385 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9bd5997e-2555-4669-8299-cccf6d045999 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:44.103612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5997e-2555-4669-8299-cccf6d045999 service nova] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:44.126810 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87a40d-7cb8-4ffe-98f5-694c561ef339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:44.127563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87a40d-7cb8-4ffe-98f5-694c561ef339 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:44.127783 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87a40d-7cb8-4ffe-98f5-694c561ef339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:44.128062 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87a40d-7cb8-4ffe-98f5-694c561ef339 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:44.133295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87a40d-7cb8-4ffe-98f5-694c561ef339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:44.138061 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87a40d-7cb8-4ffe-98f5-694c561ef339 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:44.202437 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30a036-3596-460c-a4ea-051e6817e4c7 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:44.210564 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1107/2241] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:44 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:07:44.225474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30a036-3596-460c-a4ea-051e6817e4c7 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:44.262587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87a40d-7cb8-4ffe-98f5-694c561ef339 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:44.265727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2b87a40d-7cb8-4ffe-98f5-694c561ef339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:44.448172 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3933a7-5c1f-45a6-8315-895159872ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:44.449100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3933a7-5c1f-45a6-8315-895159872ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:44.449631 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3933a7-5c1f-45a6-8315-895159872ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:44.450005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3933a7-5c1f-45a6-8315-895159872ac1 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:44.450980 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3933a7-5c1f-45a6-8315-895159872ac1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:44.454619 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3933a7-5c1f-45a6-8315-895159872ac1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:44.458728 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87a40d-7cb8-4ffe-98f5-694c561ef339 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:44.465372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13320887-fc95-4e15-bd46-43c8e2b45cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:44.467241 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13320887-fc95-4e15-bd46-43c8e2b45cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:44.467687 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13320887-fc95-4e15-bd46-43c8e2b45cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:44.468217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13320887-fc95-4e15-bd46-43c8e2b45cef admin admin] PATH_INFO: `/v3/projects/6d67bef49576408e96d7fdd3853aa7b0` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:44.470004 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13320887-fc95-4e15-bd46-43c8e2b45cef admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d67bef49576408e96d7fdd3853aa7b0)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:44.472855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13320887-fc95-4e15-bd46-43c8e2b45cef admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:44.503926 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:44.526159 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1135/2242] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:44 2022] POST /identity/v3/users => generated 293 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:44.531016 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3933a7-5c1f-45a6-8315-895159872ac1 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:44.547984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3933a7-5c1f-45a6-8315-895159872ac1 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:44.557739 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-13320887-fc95-4e15-bd46-43c8e2b45cef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d67bef49576408e96d7fdd3853aa7b0', 'request_id': 'req-13320887-fc95-4e15-bd46-43c8e2b45cef'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:44.569894 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-13320887-fc95-4e15-bd46-43c8e2b45cef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d67bef49576408e96d7fdd3853aa7b0', 'request_id': 'req-13320887-fc95-4e15-bd46-43c8e2b45cef'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:44.579155 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-13320887-fc95-4e15-bd46-43c8e2b45cef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d67bef49576408e96d7fdd3853aa7b0', 'request_id': 'req-13320887-fc95-4e15-bd46-43c8e2b45cef'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:44.594639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-13320887-fc95-4e15-bd46-43c8e2b45cef admin admin] The token cache is being invalidate because project 6d67bef49576408e96d7fdd3853aa7b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:44.595153 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-13320887-fc95-4e15-bd46-43c8e2b45cef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:44.595956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-13320887-fc95-4e15-bd46-43c8e2b45cef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:44.600456 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1108/2243] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:44 2022] DELETE /identity/v3/projects/6d67bef49576408e96d7fdd3853aa7b0 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:44.602876 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1fe45-9c6d-4dc2-ad83-4794314a3959 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:44.605745 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1fe45-9c6d-4dc2-ad83-4794314a3959 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:44.606117 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1fe45-9c6d-4dc2-ad83-4794314a3959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:44.643120 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-efb1fe45-9c6d-4dc2-ad83-4794314a3959 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:44.644925 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-efb1fe45-9c6d-4dc2-ad83-4794314a3959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:44.659443 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc2f59-bf6d-47cb-a158-9ba8bd6dc037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:44.660611 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc2f59-bf6d-47cb-a158-9ba8bd6dc037 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:44.661790 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc2f59-bf6d-47cb-a158-9ba8bd6dc037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:44.662349 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc2f59-bf6d-47cb-a158-9ba8bd6dc037 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:44.663515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc2f59-bf6d-47cb-a158-9ba8bd6dc037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:44.665916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc2f59-bf6d-47cb-a158-9ba8bd6dc037 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:44.698087 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1136/2244] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:44.699315 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc2f59-bf6d-47cb-a158-9ba8bd6dc037 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:44.701654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f7dc2f59-bf6d-47cb-a158-9ba8bd6dc037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:44.718698 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc2f59-bf6d-47cb-a158-9ba8bd6dc037 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:44.726130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9631b-d88e-40cd-85e2-f00587cf2b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:44.728280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9631b-d88e-40cd-85e2-f00587cf2b1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:44.728890 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9631b-d88e-40cd-85e2-f00587cf2b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:44.729400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9631b-d88e-40cd-85e2-f00587cf2b1a admin admin] PATH_INFO: `/v3/projects/126479dc7bc54e28906eedd13c141edf/users/bddfa4e17aaa436cb32201587d0ce2b5/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:44.756872 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1109/2245] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:44.757116 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f9631b-d88e-40cd-85e2-f00587cf2b1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=126479dc7bc54e28906eedd13c141edf, user_id=bddfa4e17aaa436cb32201587d0ce2b5, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:44.760686 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d1ee5-7982-42ad-911e-78a544b58d18 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:44.760854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d1ee5-7982-42ad-911e-78a544b58d18 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:44.761050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d1ee5-7982-42ad-911e-78a544b58d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:44.763546 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f9631b-d88e-40cd-85e2-f00587cf2b1a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:44.789595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1137/2246] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:44 2022] PUT /identity/v3/projects/126479dc7bc54e28906eedd13c141edf/users/bddfa4e17aaa436cb32201587d0ce2b5/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:44.790933 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9631b-d88e-40cd-85e2-f00587cf2b1a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:44.811071 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-fc6d1ee5-7982-42ad-911e-78a544b58d18 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:44.813615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-fc6d1ee5-7982-42ad-911e-78a544b58d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:44.929462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1110/2247] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:44.931635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d1ee5-7982-42ad-911e-78a544b58d18 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:44.944170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9631b-d88e-40cd-85e2-f00587cf2b1a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:44.953523 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d1ee5-7982-42ad-911e-78a544b58d18 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:44.957198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefd0fe-1fbd-454a-abda-cf321f413980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:44.958183 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefd0fe-1fbd-454a-abda-cf321f413980 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:44.958400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefd0fe-1fbd-454a-abda-cf321f413980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:44.958849 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefd0fe-1fbd-454a-abda-cf321f413980 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:44.959077 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f612e-3a03-4056-a6b7-36f18df41860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:44.959763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f612e-3a03-4056-a6b7-36f18df41860 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:44.959964 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefd0fe-1fbd-454a-abda-cf321f413980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:44.960107 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f612e-3a03-4056-a6b7-36f18df41860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:44.960353 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f612e-3a03-4056-a6b7-36f18df41860 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:44.961225 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f612e-3a03-4056-a6b7-36f18df41860 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:07:44.961523 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f612e-3a03-4056-a6b7-36f18df41860 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:44.962895 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefd0fe-1fbd-454a-abda-cf321f413980 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:44.963823 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f612e-3a03-4056-a6b7-36f18df41860 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:44.973296 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:44.981669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1111/2248] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:07:44.983444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f612e-3a03-4056-a6b7-36f18df41860 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:44.988381 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1138/2249] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:44.993533 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefd0fe-1fbd-454a-abda-cf321f413980 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:44.997940 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f612e-3a03-4056-a6b7-36f18df41860 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:45.003431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9930554-dfdd-4849-b1e5-20bb92756403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:45.004109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9930554-dfdd-4849-b1e5-20bb92756403 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:45.004394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9930554-dfdd-4849-b1e5-20bb92756403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:45.004807 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9930554-dfdd-4849-b1e5-20bb92756403 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:45.005681 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9930554-dfdd-4849-b1e5-20bb92756403 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:45.007764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9930554-dfdd-4849-b1e5-20bb92756403 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:45.010974 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefd0fe-1fbd-454a-abda-cf321f413980 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:45.022905 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f5e56-9128-465a-a191-3771abb4864a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:45.024396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f5e56-9128-465a-a191-3771abb4864a admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:45.024396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f5e56-9128-465a-a191-3771abb4864a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:45.024396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f5e56-9128-465a-a191-3771abb4864a admin admin] PATH_INFO: `/v3/projects/126479dc7bc54e28906eedd13c141edf/users/bddfa4e17aaa436cb32201587d0ce2b5/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:45.026645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f5e56-9128-465a-a191-3771abb4864a admin admin] RBAC: Authorizing `identity:create_grant(project_id=126479dc7bc54e28906eedd13c141edf, user_id=bddfa4e17aaa436cb32201587d0ce2b5, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:45.029945 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f5e56-9128-465a-a191-3771abb4864a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:45.052637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1139/2250] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:44 2022] PUT /identity/v3/projects/126479dc7bc54e28906eedd13c141edf/users/bddfa4e17aaa436cb32201587d0ce2b5/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:45.053238 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1112/2251] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:44 2022] POST /identity/v3/projects => generated 381 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:45.055328 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f5e56-9128-465a-a191-3771abb4864a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:45.131857 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f5e56-9128-465a-a191-3771abb4864a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:45.238914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588db930-4d84-4da1-8a6d-76660e34a39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:45.239755 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588db930-4d84-4da1-8a6d-76660e34a39d admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:45.240202 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588db930-4d84-4da1-8a6d-76660e34a39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:45.240791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588db930-4d84-4da1-8a6d-76660e34a39d admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:45.241342 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588db930-4d84-4da1-8a6d-76660e34a39d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:45.244744 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588db930-4d84-4da1-8a6d-76660e34a39d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:45.273966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1140/2252] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:45 2022] POST /identity/v3/users => generated 291 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:45.277138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9930554-dfdd-4849-b1e5-20bb92756403 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:45.290139 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9930554-dfdd-4849-b1e5-20bb92756403 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:45.296745 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e287b-fc49-4793-a93a-c3f17371dac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:45.298164 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e287b-fc49-4793-a93a-c3f17371dac9 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:45.298164 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e287b-fc49-4793-a93a-c3f17371dac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:45.298359 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e287b-fc49-4793-a93a-c3f17371dac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:45.299385 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e287b-fc49-4793-a93a-c3f17371dac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:45.308327 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e287b-fc49-4793-a93a-c3f17371dac9 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:45.313317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:45.327910 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1113/2253] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:45.331801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588db930-4d84-4da1-8a6d-76660e34a39d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:45.350466 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588db930-4d84-4da1-8a6d-76660e34a39d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:45.358148 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d2f718-0886-4b1c-8e69-679799115e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:45.359095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2f718-0886-4b1c-8e69-679799115e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:45.359714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2f718-0886-4b1c-8e69-679799115e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:45.360092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d2f718-0886-4b1c-8e69-679799115e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:45.361390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d2f718-0886-4b1c-8e69-679799115e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:45.363552 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d2f718-0886-4b1c-8e69-679799115e90 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:45.385961 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1141/2254] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:45.390652 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e287b-fc49-4793-a93a-c3f17371dac9 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:45.409702 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e287b-fc49-4793-a93a-c3f17371dac9 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:45.423184 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d768498-7b64-4c27-9ce3-dc9e122a30e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:45.424464 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d768498-7b64-4c27-9ce3-dc9e122a30e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:45.425018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d768498-7b64-4c27-9ce3-dc9e122a30e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:45.425507 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d768498-7b64-4c27-9ce3-dc9e122a30e2 admin admin] PATH_INFO: `/v3/projects/4e1e0c6b3e544d138a1c7911cf155076/users/bf0332a9562946169ff8a45bf5e6f6f3/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:45.468190 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d768498-7b64-4c27-9ce3-dc9e122a30e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e1e0c6b3e544d138a1c7911cf155076, user_id=bf0332a9562946169ff8a45bf5e6f6f3, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:45.472233 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d768498-7b64-4c27-9ce3-dc9e122a30e2 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:45.506419 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1114/2255] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:45 2022] PUT /identity/v3/projects/4e1e0c6b3e544d138a1c7911cf155076/users/bf0332a9562946169ff8a45bf5e6f6f3/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:45.511743 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c01f5-764b-4306-bdc4-b45402b212f4 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:45.512071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c01f5-764b-4306-bdc4-b45402b212f4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:45.512355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c01f5-764b-4306-bdc4-b45402b212f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:45.548848 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1e8c01f5-764b-4306-bdc4-b45402b212f4 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:45.550082 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1e8c01f5-764b-4306-bdc4-b45402b212f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:45.780894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1142/2256] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:45.829834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d768498-7b64-4c27-9ce3-dc9e122a30e2 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:45.834473 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2d768498-7b64-4c27-9ce3-dc9e122a30e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:45.955725 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d768498-7b64-4c27-9ce3-dc9e122a30e2 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:45.968172 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f5549-2652-4682-ad44-4c02815c832c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:45.969054 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f5549-2652-4682-ad44-4c02815c832c service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:45.969568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f5549-2652-4682-ad44-4c02815c832c service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:45.970100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f5549-2652-4682-ad44-4c02815c832c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:45.984600 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f5549-2652-4682-ad44-4c02815c832c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:45.986251 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f5549-2652-4682-ad44-4c02815c832c service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:46.011586 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1115/2257] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:46.113910 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c01f5-764b-4306-bdc4-b45402b212f4 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:46.115747 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1e8c01f5-764b-4306-bdc4-b45402b212f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:46.140350 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c01f5-764b-4306-bdc4-b45402b212f4 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:46.149173 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63feaf8-a98d-47db-a5c6-c3a1714fb369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:46.150130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63feaf8-a98d-47db-a5c6-c3a1714fb369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:46.150403 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63feaf8-a98d-47db-a5c6-c3a1714fb369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:46.150677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63feaf8-a98d-47db-a5c6-c3a1714fb369 admin admin] PATH_INFO: `/v3/users/56a897b3156b44829d9b954b6f5224e8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:46.152051 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63feaf8-a98d-47db-a5c6-c3a1714fb369 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56a897b3156b44829d9b954b6f5224e8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:46.154754 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63feaf8-a98d-47db-a5c6-c3a1714fb369 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:46.166501 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:46.242374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f63feaf8-a98d-47db-a5c6-c3a1714fb369 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56a897b3156b44829d9b954b6f5224e8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:46.254746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f63feaf8-a98d-47db-a5c6-c3a1714fb369 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56a897b3156b44829d9b954b6f5224e8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:46.256784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f63feaf8-a98d-47db-a5c6-c3a1714fb369 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56a897b3156b44829d9b954b6f5224e8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:46.270778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f63feaf8-a98d-47db-a5c6-c3a1714fb369 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56a897b3156b44829d9b954b6f5224e8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:46.271703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f63feaf8-a98d-47db-a5c6-c3a1714fb369 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56a897b3156b44829d9b954b6f5224e8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:46.298207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1143/2258] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:46 2022] DELETE /identity/v3/users/56a897b3156b44829d9b954b6f5224e8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:46.303634 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3661e-7e4a-4474-9a0f-1909280160a1 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:46.304231 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3661e-7e4a-4474-9a0f-1909280160a1 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:46.304660 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3661e-7e4a-4474-9a0f-1909280160a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:46.335683 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-91c3661e-7e4a-4474-9a0f-1909280160a1 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:46.337558 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-91c3661e-7e4a-4474-9a0f-1909280160a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:46.437285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1116/2259] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:46.472885 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63feaf8-a98d-47db-a5c6-c3a1714fb369 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:46.476083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f63feaf8-a98d-47db-a5c6-c3a1714fb369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:46.575838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63feaf8-a98d-47db-a5c6-c3a1714fb369 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:46.582968 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7d7c3-2dea-495c-b0b6-aa7baaf0e180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:46.585493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7d7c3-2dea-495c-b0b6-aa7baaf0e180 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:46.586680 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7d7c3-2dea-495c-b0b6-aa7baaf0e180 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:46.588149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7d7c3-2dea-495c-b0b6-aa7baaf0e180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:46.612601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a7d7c3-2dea-495c-b0b6-aa7baaf0e180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:46.614215 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a7d7c3-2dea-495c-b0b6-aa7baaf0e180 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:46.639660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1144/2260] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:47.609461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38671b2-3c05-4ba7-904e-9c6b5986c456 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:47.610661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38671b2-3c05-4ba7-904e-9c6b5986c456 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:47.611137 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38671b2-3c05-4ba7-904e-9c6b5986c456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:47.672685 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c38671b2-3c05-4ba7-904e-9c6b5986c456 None None] MFA Rules not processed for user `bddfa4e17aaa436cb32201587d0ce2b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:47.675016 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c38671b2-3c05-4ba7-904e-9c6b5986c456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:47.806084 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7d7c3-2dea-495c-b0b6-aa7baaf0e180 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:47.809879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-87a7d7c3-2dea-495c-b0b6-aa7baaf0e180 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:47.851279 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7d7c3-2dea-495c-b0b6-aa7baaf0e180 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:47.864979 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08244331-f9d7-47ae-8e1b-f1c2b36c8841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:47.867439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08244331-f9d7-47ae-8e1b-f1c2b36c8841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:47.867820 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08244331-f9d7-47ae-8e1b-f1c2b36c8841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:47.868199 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08244331-f9d7-47ae-8e1b-f1c2b36c8841 admin admin] PATH_INFO: `/v3/projects/ae0ab5bafa0547738a0214a2f0fb750b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:47.870616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08244331-f9d7-47ae-8e1b-f1c2b36c8841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae0ab5bafa0547738a0214a2f0fb750b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:47.874163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08244331-f9d7-47ae-8e1b-f1c2b36c8841 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:47.899049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1117/2261] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:47.908062 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38671b2-3c05-4ba7-904e-9c6b5986c456 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:47.910201 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c38671b2-3c05-4ba7-904e-9c6b5986c456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:47.918742 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:47.987043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-08244331-f9d7-47ae-8e1b-f1c2b36c8841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae0ab5bafa0547738a0214a2f0fb750b', 'request_id': 'req-08244331-f9d7-47ae-8e1b-f1c2b36c8841'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:48.016769 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-08244331-f9d7-47ae-8e1b-f1c2b36c8841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae0ab5bafa0547738a0214a2f0fb750b', 'request_id': 'req-08244331-f9d7-47ae-8e1b-f1c2b36c8841'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:48.035367 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-08244331-f9d7-47ae-8e1b-f1c2b36c8841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae0ab5bafa0547738a0214a2f0fb750b', 'request_id': 'req-08244331-f9d7-47ae-8e1b-f1c2b36c8841'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:48.070537 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-08244331-f9d7-47ae-8e1b-f1c2b36c8841 admin admin] The token cache is being invalidate because project ae0ab5bafa0547738a0214a2f0fb750b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:48.071152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-08244331-f9d7-47ae-8e1b-f1c2b36c8841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:48.072676 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-08244331-f9d7-47ae-8e1b-f1c2b36c8841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:48.074649 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38671b2-3c05-4ba7-904e-9c6b5986c456 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:48.075796 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1145/2262] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:47 2022] DELETE /identity/v3/projects/ae0ab5bafa0547738a0214a2f0fb750b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:07:48.079476 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08244331-f9d7-47ae-8e1b-f1c2b36c8841 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:48.081423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-08244331-f9d7-47ae-8e1b-f1c2b36c8841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:48.094814 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:48.186560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08244331-f9d7-47ae-8e1b-f1c2b36c8841 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:48.195063 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539df30-bc25-4cc6-8ff1-93eedd3c2448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:48.196605 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539df30-bc25-4cc6-8ff1-93eedd3c2448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:48.197214 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539df30-bc25-4cc6-8ff1-93eedd3c2448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:48.197756 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539df30-bc25-4cc6-8ff1-93eedd3c2448 admin admin] PATH_INFO: `/v3/users/a8b099bf715e40948d9def8f7e43f3b0` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:48.200642 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3539df30-bc25-4cc6-8ff1-93eedd3c2448 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8b099bf715e40948d9def8f7e43f3b0)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:48.203056 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3539df30-bc25-4cc6-8ff1-93eedd3c2448 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:48.219018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4237e-553b-4aa4-8523-5b2ff05dd78d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:48.219694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4237e-553b-4aa4-8523-5b2ff05dd78d service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:48.219963 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4237e-553b-4aa4-8523-5b2ff05dd78d service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:48.220233 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4237e-553b-4aa4-8523-5b2ff05dd78d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:48.222371 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-13a4237e-553b-4aa4-8523-5b2ff05dd78d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:48.316706 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3539df30-bc25-4cc6-8ff1-93eedd3c2448 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8b099bf715e40948d9def8f7e43f3b0'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:48.337761 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3539df30-bc25-4cc6-8ff1-93eedd3c2448 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8b099bf715e40948d9def8f7e43f3b0'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:48.339813 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3539df30-bc25-4cc6-8ff1-93eedd3c2448 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8b099bf715e40948d9def8f7e43f3b0'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:48.358805 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3539df30-bc25-4cc6-8ff1-93eedd3c2448 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8b099bf715e40948d9def8f7e43f3b0'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:48.359908 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3539df30-bc25-4cc6-8ff1-93eedd3c2448 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8b099bf715e40948d9def8f7e43f3b0'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:48.367733 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4237e-553b-4aa4-8523-5b2ff05dd78d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:48.369559 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4237e-553b-4aa4-8523-5b2ff05dd78d service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:48.400102 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1118/2263] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:48.406559 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1146/2264] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:48 2022] DELETE /identity/v3/users/a8b099bf715e40948d9def8f7e43f3b0 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:48.704211 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f0327-8158-4cf9-8ec7-16face7d2a58 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:48.704770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f0327-8158-4cf9-8ec7-16face7d2a58 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:48.704995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f0327-8158-4cf9-8ec7-16face7d2a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:48.744798 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-255f0327-8158-4cf9-8ec7-16face7d2a58 None None] MFA Rules not processed for user `744eb9d4e7084b7e9cae36d5de67f0f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:48.746153 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-255f0327-8158-4cf9-8ec7-16face7d2a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:48.763240 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029fc30-f628-4cdb-b263-57d42eeb4fb5 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:48.763917 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029fc30-f628-4cdb-b263-57d42eeb4fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:48.764538 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029fc30-f628-4cdb-b263-57d42eeb4fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:48.842860 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-2029fc30-f628-4cdb-b263-57d42eeb4fb5 None None] MFA Rules not processed for user `bf0332a9562946169ff8a45bf5e6f6f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:48.846035 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2029fc30-f628-4cdb-b263-57d42eeb4fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:48.906902 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1119/2265] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:48.993561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1147/2266] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:49.007755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f0327-8158-4cf9-8ec7-16face7d2a58 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:49.009332 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-255f0327-8158-4cf9-8ec7-16face7d2a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:49.051736 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:49.134937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f0327-8158-4cf9-8ec7-16face7d2a58 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:49.141123 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb9a5b-9286-42a4-8e40-356c4f1d9773 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:49.142309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb9a5b-9286-42a4-8e40-356c4f1d9773 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:49.142741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb9a5b-9286-42a4-8e40-356c4f1d9773 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:49.143189 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb9a5b-9286-42a4-8e40-356c4f1d9773 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:49.156510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb9a5b-9286-42a4-8e40-356c4f1d9773 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:49.161167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb9a5b-9286-42a4-8e40-356c4f1d9773 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:49.174252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1120/2267] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:07:49.670029 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029fc30-f628-4cdb-b263-57d42eeb4fb5 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:49.674638 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2029fc30-f628-4cdb-b263-57d42eeb4fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:49.719178 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:49.910540 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029fc30-f628-4cdb-b263-57d42eeb4fb5 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:49.920212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090344d-af91-44e3-a093-ed0cd0b3f0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:49.921727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090344d-af91-44e3-a093-ed0cd0b3f0ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:49.922370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090344d-af91-44e3-a093-ed0cd0b3f0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:49.922776 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090344d-af91-44e3-a093-ed0cd0b3f0ff admin admin] PATH_INFO: `/v3/projects/3fab615f4c894dc8927dd9be1f30dcd6` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:49.925142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090344d-af91-44e3-a093-ed0cd0b3f0ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fab615f4c894dc8927dd9be1f30dcd6)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:49.944337 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090344d-af91-44e3-a093-ed0cd0b3f0ff admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:50.117555 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb9a5b-9286-42a4-8e40-356c4f1d9773 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:50.121563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9afb9a5b-9286-42a4-8e40-356c4f1d9773 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:50.130776 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9090344d-af91-44e3-a093-ed0cd0b3f0ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fab615f4c894dc8927dd9be1f30dcd6', 'request_id': 'req-9090344d-af91-44e3-a093-ed0cd0b3f0ff'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:50.138900 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9090344d-af91-44e3-a093-ed0cd0b3f0ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fab615f4c894dc8927dd9be1f30dcd6', 'request_id': 'req-9090344d-af91-44e3-a093-ed0cd0b3f0ff'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:50.148781 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9090344d-af91-44e3-a093-ed0cd0b3f0ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fab615f4c894dc8927dd9be1f30dcd6', 'request_id': 'req-9090344d-af91-44e3-a093-ed0cd0b3f0ff'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:50.149102 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb9a5b-9286-42a4-8e40-356c4f1d9773 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:50.188996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9090344d-af91-44e3-a093-ed0cd0b3f0ff admin admin] The token cache is being invalidate because project 3fab615f4c894dc8927dd9be1f30dcd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:50.189799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9090344d-af91-44e3-a093-ed0cd0b3f0ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:50.193206 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9090344d-af91-44e3-a093-ed0cd0b3f0ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:50.222043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1148/2268] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:49 2022] DELETE /identity/v3/projects/3fab615f4c894dc8927dd9be1f30dcd6 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:50.223273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090344d-af91-44e3-a093-ed0cd0b3f0ff admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:50.234886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9090344d-af91-44e3-a093-ed0cd0b3f0ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:50.275970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c26068-ea30-4ac5-a8e9-4c8630c2d3bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:50.277363 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c26068-ea30-4ac5-a8e9-4c8630c2d3bc service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:50.277881 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c26068-ea30-4ac5-a8e9-4c8630c2d3bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:50.278350 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c26068-ea30-4ac5-a8e9-4c8630c2d3bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:50.281470 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-16c26068-ea30-4ac5-a8e9-4c8630c2d3bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:50.392351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090344d-af91-44e3-a093-ed0cd0b3f0ff admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:50.405025 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0077e08-6645-4a7c-9ee1-b5574159cebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:50.407651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0077e08-6645-4a7c-9ee1-b5574159cebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:50.407992 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0077e08-6645-4a7c-9ee1-b5574159cebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:50.418952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0077e08-6645-4a7c-9ee1-b5574159cebc admin admin] PATH_INFO: `/v3/users/1f4120288ec44586935a754216dfac72` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:50.427481 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0077e08-6645-4a7c-9ee1-b5574159cebc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f4120288ec44586935a754216dfac72)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:50.431737 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0077e08-6645-4a7c-9ee1-b5574159cebc admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:50.503283 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c26068-ea30-4ac5-a8e9-4c8630c2d3bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:50.505852 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c26068-ea30-4ac5-a8e9-4c8630c2d3bc service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:50.533293 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1121/2269] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:50 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:07:50.543693 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b0077e08-6645-4a7c-9ee1-b5574159cebc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f4120288ec44586935a754216dfac72'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:50.552124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:50.556071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b0077e08-6645-4a7c-9ee1-b5574159cebc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f4120288ec44586935a754216dfac72'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:50.564728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b0077e08-6645-4a7c-9ee1-b5574159cebc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f4120288ec44586935a754216dfac72'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:50.575674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b0077e08-6645-4a7c-9ee1-b5574159cebc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f4120288ec44586935a754216dfac72'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:50.576703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-b0077e08-6645-4a7c-9ee1-b5574159cebc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f4120288ec44586935a754216dfac72'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:50.614067 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1149/2270] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:07:50 2022] DELETE /identity/v3/users/1f4120288ec44586935a754216dfac72 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:50.846018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0330a-f6ae-46bf-af1b-b1fe46860b08 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:50.851795 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0330a-f6ae-46bf-af1b-b1fe46860b08 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:50.852096 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0330a-f6ae-46bf-af1b-b1fe46860b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:50.924683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db98249-21ad-46e7-beba-ebff1d39c86d None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:50.924683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db98249-21ad-46e7-beba-ebff1d39c86d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:50.924683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db98249-21ad-46e7-beba-ebff1d39c86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:50.929355 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-87a0330a-f6ae-46bf-af1b-b1fe46860b08 None None] MFA Rules not processed for user `5bd682806c864805a3b410411839ca99`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:50.931138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-87a0330a-f6ae-46bf-af1b-b1fe46860b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:50.963688 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-9db98249-21ad-46e7-beba-ebff1d39c86d None None] MFA Rules not processed for user `bf0332a9562946169ff8a45bf5e6f6f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:50.966625 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-9db98249-21ad-46e7-beba-ebff1d39c86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:51.062349 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:51.068975 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1122/2271] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:51.085965 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0330a-f6ae-46bf-af1b-b1fe46860b08 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:51.089877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-87a0330a-f6ae-46bf-af1b-b1fe46860b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:51.132239 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1150/2272] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:07:51.222568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0330a-f6ae-46bf-af1b-b1fe46860b08 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:51.229674 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80df9f9-da0a-43a6-84dd-117862210b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:51.230573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80df9f9-da0a-43a6-84dd-117862210b74 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:51.231114 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80df9f9-da0a-43a6-84dd-117862210b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:51.231560 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80df9f9-da0a-43a6-84dd-117862210b74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:51.232561 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80df9f9-da0a-43a6-84dd-117862210b74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:51.234833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80df9f9-da0a-43a6-84dd-117862210b74 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:51.271943 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db98249-21ad-46e7-beba-ebff1d39c86d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:51.272670 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1123/2273] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:51 2022] POST /identity/v3/projects => generated 373 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:51.287998 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db98249-21ad-46e7-beba-ebff1d39c86d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:51.363988 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80df9f9-da0a-43a6-84dd-117862210b74 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:51.367207 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c80df9f9-da0a-43a6-84dd-117862210b74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:51.434034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1eadf-11f2-49a2-abf1-d3c9718ffa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:51.434986 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1eadf-11f2-49a2-abf1-d3c9718ffa9f admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:51.435357 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1eadf-11f2-49a2-abf1-d3c9718ffa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:51.435728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d1eadf-11f2-49a2-abf1-d3c9718ffa9f admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:51.436605 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1eadf-11f2-49a2-abf1-d3c9718ffa9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:51.438939 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d1eadf-11f2-49a2-abf1-d3c9718ffa9f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:51.462020 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80df9f9-da0a-43a6-84dd-117862210b74 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:51.465879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1151/2274] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:51 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:07:51.467047 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1eadf-11f2-49a2-abf1-d3c9718ffa9f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:51.477425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc48261-e1cc-483f-bd8f-acbde4175483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:51.478472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc48261-e1cc-483f-bd8f-acbde4175483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:51.478897 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc48261-e1cc-483f-bd8f-acbde4175483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:51.478989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d1eadf-11f2-49a2-abf1-d3c9718ffa9f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:51.479219 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc48261-e1cc-483f-bd8f-acbde4175483 admin admin] PATH_INFO: `/v3/projects/cf5c5d3011f1434ba00e3f79078541bd` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:51.481074 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc48261-e1cc-483f-bd8f-acbde4175483 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf5c5d3011f1434ba00e3f79078541bd)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:51.483312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc48261-e1cc-483f-bd8f-acbde4175483 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:51.486480 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ff72a-5d13-4242-9c8e-76239c914848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:51.488200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ff72a-5d13-4242-9c8e-76239c914848 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:51.488709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ff72a-5d13-4242-9c8e-76239c914848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:51.489136 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ff72a-5d13-4242-9c8e-76239c914848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:51.490858 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ff72a-5d13-4242-9c8e-76239c914848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:51.492747 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ff72a-5d13-4242-9c8e-76239c914848 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:51.511472 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1152/2275] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:51.512298 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ff72a-5d13-4242-9c8e-76239c914848 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:51.529498 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ff72a-5d13-4242-9c8e-76239c914848 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:51.541175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9c1dd-fd2a-43e9-a53d-ed32ddeeed23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:51.542124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9c1dd-fd2a-43e9-a53d-ed32ddeeed23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:51.542403 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9c1dd-fd2a-43e9-a53d-ed32ddeeed23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:51.543114 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9c1dd-fd2a-43e9-a53d-ed32ddeeed23 admin admin] PATH_INFO: `/v3/projects/45fcc90f2d094bffa9fc393df4548413/users/5ac532f064934130afc11fff87c73617/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:51.567916 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ebc48261-e1cc-483f-bd8f-acbde4175483 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf5c5d3011f1434ba00e3f79078541bd', 'request_id': 'req-ebc48261-e1cc-483f-bd8f-acbde4175483'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:51.572173 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9c1dd-fd2a-43e9-a53d-ed32ddeeed23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45fcc90f2d094bffa9fc393df4548413, user_id=5ac532f064934130afc11fff87c73617, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:51.574647 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ebc48261-e1cc-483f-bd8f-acbde4175483 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf5c5d3011f1434ba00e3f79078541bd', 'request_id': 'req-ebc48261-e1cc-483f-bd8f-acbde4175483'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:51.575488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9c1dd-fd2a-43e9-a53d-ed32ddeeed23 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:51.582407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ebc48261-e1cc-483f-bd8f-acbde4175483 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf5c5d3011f1434ba00e3f79078541bd', 'request_id': 'req-ebc48261-e1cc-483f-bd8f-acbde4175483'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:51.600223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ebc48261-e1cc-483f-bd8f-acbde4175483 admin admin] The token cache is being invalidate because project cf5c5d3011f1434ba00e3f79078541bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:07:51.600223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ebc48261-e1cc-483f-bd8f-acbde4175483 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:51.600223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ebc48261-e1cc-483f-bd8f-acbde4175483 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:07:51.600846 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1153/2276] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:51 2022] PUT /identity/v3/projects/45fcc90f2d094bffa9fc393df4548413/users/5ac532f064934130afc11fff87c73617/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:51.602067 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9c1dd-fd2a-43e9-a53d-ed32ddeeed23 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:51.604005 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7fb9c1dd-fd2a-43e9-a53d-ed32ddeeed23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:51.607418 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1124/2277] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:07:51 2022] DELETE /identity/v3/projects/cf5c5d3011f1434ba00e3f79078541bd => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:07:51.608852 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46626990-fc65-4499-9c11-05931ecbd97e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:51.609156 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46626990-fc65-4499-9c11-05931ecbd97e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:51.609459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46626990-fc65-4499-9c11-05931ecbd97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:51.629778 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-46626990-fc65-4499-9c11-05931ecbd97e None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:51.631074 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-46626990-fc65-4499-9c11-05931ecbd97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:51.643244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:51.678266 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:51.758553 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1125/2278] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:51.770479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdd6aa-847f-412b-a205-5a6284832dd2 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:51.771247 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdd6aa-847f-412b-a205-5a6284832dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:51.777313 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdd6aa-847f-412b-a205-5a6284832dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:51.790572 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9c1dd-fd2a-43e9-a53d-ed32ddeeed23 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:51.796235 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294fa2d2-4fc9-4ba1-89b7-20307a578c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:51.797231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fa2d2-4fc9-4ba1-89b7-20307a578c7c admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:51.797605 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fa2d2-4fc9-4ba1-89b7-20307a578c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:51.797882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fa2d2-4fc9-4ba1-89b7-20307a578c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:51.798672 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294fa2d2-4fc9-4ba1-89b7-20307a578c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:51.801356 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294fa2d2-4fc9-4ba1-89b7-20307a578c7c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:51.820492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1154/2279] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:51.822026 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294fa2d2-4fc9-4ba1-89b7-20307a578c7c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:51.833821 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294fa2d2-4fc9-4ba1-89b7-20307a578c7c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:51.843758 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35685e81-41ff-44c1-a6f7-1a7780209a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:51.844316 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-fcbdd6aa-847f-412b-a205-5a6284832dd2 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:51.845717 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-fcbdd6aa-847f-412b-a205-5a6284832dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:51.845855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35685e81-41ff-44c1-a6f7-1a7780209a1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:51.847122 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35685e81-41ff-44c1-a6f7-1a7780209a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:51.851371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35685e81-41ff-44c1-a6f7-1a7780209a1a admin admin] PATH_INFO: `/v3/projects/45fcc90f2d094bffa9fc393df4548413/users/5ac532f064934130afc11fff87c73617/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:51.853634 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35685e81-41ff-44c1-a6f7-1a7780209a1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=45fcc90f2d094bffa9fc393df4548413, user_id=5ac532f064934130afc11fff87c73617, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:51.866974 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35685e81-41ff-44c1-a6f7-1a7780209a1a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:51.872392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1126/2280] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:51.878113 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdd6aa-847f-412b-a205-5a6284832dd2 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:51.900497 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdd6aa-847f-412b-a205-5a6284832dd2 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:51.905041 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1155/2281] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:51 2022] PUT /identity/v3/projects/45fcc90f2d094bffa9fc393df4548413/users/5ac532f064934130afc11fff87c73617/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:07:52.108875 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56f4cb-a0f3-4e61-bc55-3f21d50ff6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:52.109295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56f4cb-a0f3-4e61-bc55-3f21d50ff6db admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:52.109588 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56f4cb-a0f3-4e61-bc55-3f21d50ff6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:52.110174 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56f4cb-a0f3-4e61-bc55-3f21d50ff6db admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:52.110917 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56f4cb-a0f3-4e61-bc55-3f21d50ff6db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:07:52.111222 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56f4cb-a0f3-4e61-bc55-3f21d50ff6db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:52.117550 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56f4cb-a0f3-4e61-bc55-3f21d50ff6db admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:52.151453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1127/2282] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:07:52.156013 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35685e81-41ff-44c1-a6f7-1a7780209a1a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:52.172345 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35685e81-41ff-44c1-a6f7-1a7780209a1a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:52.182025 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb627c3-5749-4953-a11a-24a9b3343fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:52.199846 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb627c3-5749-4953-a11a-24a9b3343fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:52.200616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb627c3-5749-4953-a11a-24a9b3343fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:52.200916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb627c3-5749-4953-a11a-24a9b3343fc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:52.203570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb627c3-5749-4953-a11a-24a9b3343fc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:52.209445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb627c3-5749-4953-a11a-24a9b3343fc7 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:52.280336 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1156/2283] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:07:52 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:52.286541 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56f4cb-a0f3-4e61-bc55-3f21d50ff6db admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:52.302024 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56f4cb-a0f3-4e61-bc55-3f21d50ff6db admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:52.453984 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3077e-c9e1-476d-ae88-96e0865d9c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:52.454684 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3077e-c9e1-476d-ae88-96e0865d9c8d admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:52.455076 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3077e-c9e1-476d-ae88-96e0865d9c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:52.455292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3077e-c9e1-476d-ae88-96e0865d9c8d admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:52.456002 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3077e-c9e1-476d-ae88-96e0865d9c8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:52.458006 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3077e-c9e1-476d-ae88-96e0865d9c8d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:52.527162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1128/2284] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:07:52 2022] POST /identity/v3/users => generated 291 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:07:52.530983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb627c3-5749-4953-a11a-24a9b3343fc7 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:52.570534 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb627c3-5749-4953-a11a-24a9b3343fc7 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:52.581759 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d6e21-14d1-4a39-b57d-a2f6005f08c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:52.589838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d6e21-14d1-4a39-b57d-a2f6005f08c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:52.590241 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d6e21-14d1-4a39-b57d-a2f6005f08c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:52.590523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d6e21-14d1-4a39-b57d-a2f6005f08c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:52.591415 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d6e21-14d1-4a39-b57d-a2f6005f08c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:52.600647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d6e21-14d1-4a39-b57d-a2f6005f08c8 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:52.626918 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1157/2285] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:07:52.632056 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3077e-c9e1-476d-ae88-96e0865d9c8d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:52.651088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3077e-c9e1-476d-ae88-96e0865d9c8d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:52.672273 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019683d-ea30-4045-acbb-9662d8b61687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:52.673627 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019683d-ea30-4045-acbb-9662d8b61687 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:52.673910 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019683d-ea30-4045-acbb-9662d8b61687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:52.674230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019683d-ea30-4045-acbb-9662d8b61687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:52.681333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019683d-ea30-4045-acbb-9662d8b61687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:52.689798 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019683d-ea30-4045-acbb-9662d8b61687 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:52.732375 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d6e21-14d1-4a39-b57d-a2f6005f08c8 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:52.739195 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1129/2286] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:07:52.750198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d6e21-14d1-4a39-b57d-a2f6005f08c8 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:52.760346 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adeb8ef-de72-4a9a-908d-73bfb75681c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:52.761457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adeb8ef-de72-4a9a-908d-73bfb75681c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:52.761766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adeb8ef-de72-4a9a-908d-73bfb75681c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:52.762047 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adeb8ef-de72-4a9a-908d-73bfb75681c4 admin admin] PATH_INFO: `/v3/projects/5940d706adcb48efb7aee52ca85813a2/users/bb02f41c395a403bba06331012f2f977/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:52.815729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adeb8ef-de72-4a9a-908d-73bfb75681c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5940d706adcb48efb7aee52ca85813a2, user_id=bb02f41c395a403bba06331012f2f977, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:52.819486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adeb8ef-de72-4a9a-908d-73bfb75681c4 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:52.868830 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1158/2287] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:07:52 2022] PUT /identity/v3/projects/5940d706adcb48efb7aee52ca85813a2/users/bb02f41c395a403bba06331012f2f977/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:07:52.870895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c24797-b2a9-4d91-8a2d-826242189020 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:52.871290 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c24797-b2a9-4d91-8a2d-826242189020 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:52.871672 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c24797-b2a9-4d91-8a2d-826242189020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:52.934755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a1c24797-b2a9-4d91-8a2d-826242189020 None None] MFA Rules not processed for user `bb02f41c395a403bba06331012f2f977`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:52.936113 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a1c24797-b2a9-4d91-8a2d-826242189020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:52.964789 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:07:53.080831 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1130/2288] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:07:53.089863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adeb8ef-de72-4a9a-908d-73bfb75681c4 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:07:53.092725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7adeb8ef-de72-4a9a-908d-73bfb75681c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:53.170584 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adeb8ef-de72-4a9a-908d-73bfb75681c4 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:07:53.176657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981d11d-aaa3-4c77-af0c-6c9fc40090d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:07:53.180308 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981d11d-aaa3-4c77-af0c-6c9fc40090d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:53.180869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981d11d-aaa3-4c77-af0c-6c9fc40090d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:53.181346 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981d11d-aaa3-4c77-af0c-6c9fc40090d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:53.199409 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981d11d-aaa3-4c77-af0c-6c9fc40090d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:07:53.201218 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981d11d-aaa3-4c77-af0c-6c9fc40090d8 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:07:53.215966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1159/2289] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:07:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:07:59.794802 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a2f09-b7da-435c-a074-d5b70a7bd5ba None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:07:59.795387 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a2f09-b7da-435c-a074-d5b70a7bd5ba None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:07:59.795487 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a2f09-b7da-435c-a074-d5b70a7bd5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:07:59.839261 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ae1a2f09-b7da-435c-a074-d5b70a7bd5ba None None] MFA Rules not processed for user `5ac532f064934130afc11fff87c73617`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:07:59.840592 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ae1a2f09-b7da-435c-a074-d5b70a7bd5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:07:59.987749 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1131/2290] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:07:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:00.073917 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981d11d-aaa3-4c77-af0c-6c9fc40090d8 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:00.075925 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e981d11d-aaa3-4c77-af0c-6c9fc40090d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:00.164351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:00.215261 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981d11d-aaa3-4c77-af0c-6c9fc40090d8 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:00.226640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9f825-9575-46d6-ba97-b13aa2aba897 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:00.227662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9f825-9575-46d6-ba97-b13aa2aba897 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:00.228317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9f825-9575-46d6-ba97-b13aa2aba897 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:00.228898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9f825-9575-46d6-ba97-b13aa2aba897 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:00.230872 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f3c9f825-9575-46d6-ba97-b13aa2aba897 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:00.315945 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c9f825-9575-46d6-ba97-b13aa2aba897 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:00.318279 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c9f825-9575-46d6-ba97-b13aa2aba897 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:00.332691 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1160/2291] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:08:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:08:02.026239 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a2f09-b7da-435c-a074-d5b70a7bd5ba None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:02.155896 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a2f09-b7da-435c-a074-d5b70a7bd5ba None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:02.165884 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00792ed-f2f2-4744-ab88-f66d767a2594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:02.166867 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00792ed-f2f2-4744-ab88-f66d767a2594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:02.167228 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00792ed-f2f2-4744-ab88-f66d767a2594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:02.167404 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00792ed-f2f2-4744-ab88-f66d767a2594 admin admin] PATH_INFO: `/v3/users/bb02f41c395a403bba06331012f2f977` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:02.171580 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00792ed-f2f2-4744-ab88-f66d767a2594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb02f41c395a403bba06331012f2f977)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:02.174021 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00792ed-f2f2-4744-ab88-f66d767a2594 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:02.258152 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:02.273038 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e00792ed-f2f2-4744-ab88-f66d767a2594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb02f41c395a403bba06331012f2f977'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:02.275057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e00792ed-f2f2-4744-ab88-f66d767a2594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb02f41c395a403bba06331012f2f977'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:02.319518 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e00792ed-f2f2-4744-ab88-f66d767a2594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb02f41c395a403bba06331012f2f977'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:02.335440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e00792ed-f2f2-4744-ab88-f66d767a2594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb02f41c395a403bba06331012f2f977'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:02.337371 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e00792ed-f2f2-4744-ab88-f66d767a2594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb02f41c395a403bba06331012f2f977'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:02.371410 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1132/2292] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:08:02 2022] DELETE /identity/v3/users/bb02f41c395a403bba06331012f2f977 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:02.372925 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572fcfe-e228-49c0-aab7-241b6aafd6fc None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:02.373741 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572fcfe-e228-49c0-aab7-241b6aafd6fc None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:02.374161 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572fcfe-e228-49c0-aab7-241b6aafd6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:02.434199 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b572fcfe-e228-49c0-aab7-241b6aafd6fc None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:02.435635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b572fcfe-e228-49c0-aab7-241b6aafd6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:02.437838 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a945f-5905-491c-b041-e2540897d5fb None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:02.438147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a945f-5905-491c-b041-e2540897d5fb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:02.438422 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a945f-5905-491c-b041-e2540897d5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:02.460660 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-319a945f-5905-491c-b041-e2540897d5fb None None] MFA Rules not processed for user `5bd682806c864805a3b410411839ca99`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:02.461905 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-319a945f-5905-491c-b041-e2540897d5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:02.553282 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1133/2293] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:08:02.575933 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1161/2294] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:02.586494 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a945f-5905-491c-b041-e2540897d5fb None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:02.588186 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-319a945f-5905-491c-b041-e2540897d5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:02.713083 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a945f-5905-491c-b041-e2540897d5fb None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:02.718445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedbbcf-d166-4050-8d07-0ba776337492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:02.719201 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedbbcf-d166-4050-8d07-0ba776337492 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:02.719490 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedbbcf-d166-4050-8d07-0ba776337492 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:02.719933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedbbcf-d166-4050-8d07-0ba776337492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:02.734227 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedbbcf-d166-4050-8d07-0ba776337492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:02.736433 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedbbcf-d166-4050-8d07-0ba776337492 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:02.786013 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1134/2295] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:08:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:08:03.841712 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572fcfe-e228-49c0-aab7-241b6aafd6fc None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:03.843601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b572fcfe-e228-49c0-aab7-241b6aafd6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:03.874088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572fcfe-e228-49c0-aab7-241b6aafd6fc None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:03.894562 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d0c82-6ffe-444b-a19d-00e81899769a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:03.899421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d0c82-6ffe-444b-a19d-00e81899769a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:03.903791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d0c82-6ffe-444b-a19d-00e81899769a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:03.906955 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d0c82-6ffe-444b-a19d-00e81899769a admin admin] PATH_INFO: `/v3/projects/5940d706adcb48efb7aee52ca85813a2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:03.916867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d0c82-6ffe-444b-a19d-00e81899769a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5940d706adcb48efb7aee52ca85813a2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:03.928027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d0c82-6ffe-444b-a19d-00e81899769a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:04.083057 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-078d0c82-6ffe-444b-a19d-00e81899769a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5940d706adcb48efb7aee52ca85813a2', 'request_id': 'req-078d0c82-6ffe-444b-a19d-00e81899769a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:04.108934 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-078d0c82-6ffe-444b-a19d-00e81899769a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5940d706adcb48efb7aee52ca85813a2', 'request_id': 'req-078d0c82-6ffe-444b-a19d-00e81899769a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:04.116918 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-078d0c82-6ffe-444b-a19d-00e81899769a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5940d706adcb48efb7aee52ca85813a2', 'request_id': 'req-078d0c82-6ffe-444b-a19d-00e81899769a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:04.148032 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-078d0c82-6ffe-444b-a19d-00e81899769a admin admin] The token cache is being invalidate because project 5940d706adcb48efb7aee52ca85813a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:08:04.148632 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-078d0c82-6ffe-444b-a19d-00e81899769a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:04.149560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-078d0c82-6ffe-444b-a19d-00e81899769a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:04.153062 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1162/2296] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:08:03 2022] DELETE /identity/v3/projects/5940d706adcb48efb7aee52ca85813a2 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:04.158276 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645126f6-7e2c-4274-8a9d-2e763ac056e5 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:04.158785 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645126f6-7e2c-4274-8a9d-2e763ac056e5 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:04.158986 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645126f6-7e2c-4274-8a9d-2e763ac056e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:04.221325 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-645126f6-7e2c-4274-8a9d-2e763ac056e5 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:04.229084 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-645126f6-7e2c-4274-8a9d-2e763ac056e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:04.404001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1135/2297] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:04.414313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7d5b4-1220-4389-a628-194cf1f56ee4 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:04.414609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7d5b4-1220-4389-a628-194cf1f56ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:04.414890 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7d5b4-1220-4389-a628-194cf1f56ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:04.451712 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e7e7d5b4-1220-4389-a628-194cf1f56ee4 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:04.454546 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e7e7d5b4-1220-4389-a628-194cf1f56ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:04.478463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1163/2298] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:04.482806 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645126f6-7e2c-4274-8a9d-2e763ac056e5 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:04.491199 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:04.499220 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645126f6-7e2c-4274-8a9d-2e763ac056e5 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:04.516198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a21f45-0b4e-44c0-b168-d7053a30e69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:04.518019 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a21f45-0b4e-44c0-b168-d7053a30e69c admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:04.518475 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a21f45-0b4e-44c0-b168-d7053a30e69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:04.518905 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a21f45-0b4e-44c0-b168-d7053a30e69c admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:04.519847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a21f45-0b4e-44c0-b168-d7053a30e69c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:08:04.520306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a21f45-0b4e-44c0-b168-d7053a30e69c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:04.528809 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a21f45-0b4e-44c0-b168-d7053a30e69c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:04.554786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1136/2299] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:08:04.558993 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7d5b4-1220-4389-a628-194cf1f56ee4 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:04.578481 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7d5b4-1220-4389-a628-194cf1f56ee4 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:04.588189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272266a-b966-46d5-b381-89ab6d362b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:04.589276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272266a-b966-46d5-b381-89ab6d362b58 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:04.589771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272266a-b966-46d5-b381-89ab6d362b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:04.590300 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272266a-b966-46d5-b381-89ab6d362b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:04.591243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272266a-b966-46d5-b381-89ab6d362b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:04.593652 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272266a-b966-46d5-b381-89ab6d362b58 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:04.608605 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:04.663756 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1164/2300] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:08:04 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:08:04.667899 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a21f45-0b4e-44c0-b168-d7053a30e69c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:04.687135 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a21f45-0b4e-44c0-b168-d7053a30e69c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:04.894862 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247640d-7c25-4503-84cf-587ce26eac2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:04.896131 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247640d-7c25-4503-84cf-587ce26eac2a admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:04.896439 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247640d-7c25-4503-84cf-587ce26eac2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:04.896951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247640d-7c25-4503-84cf-587ce26eac2a admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:04.897403 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247640d-7c25-4503-84cf-587ce26eac2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:04.899386 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247640d-7c25-4503-84cf-587ce26eac2a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:04.929306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1137/2301] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:08:04 2022] POST /identity/v3/users => generated 288 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:08:04.933542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272266a-b966-46d5-b381-89ab6d362b58 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:04.948826 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272266a-b966-46d5-b381-89ab6d362b58 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:04.954846 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac7f4c-57ad-436c-84f1-d347c87a8670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:04.955493 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac7f4c-57ad-436c-84f1-d347c87a8670 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:04.955946 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac7f4c-57ad-436c-84f1-d347c87a8670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:04.956350 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac7f4c-57ad-436c-84f1-d347c87a8670 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:04.957290 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac7f4c-57ad-436c-84f1-d347c87a8670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:04.959024 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac7f4c-57ad-436c-84f1-d347c87a8670 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:04.972595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1165/2302] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:08:04.976867 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247640d-7c25-4503-84cf-587ce26eac2a admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:04.992669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247640d-7c25-4503-84cf-587ce26eac2a admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:05.016123 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3308ba30-3954-4f04-9c28-e0dd587b15ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:05.020339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308ba30-3954-4f04-9c28-e0dd587b15ec admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:05.020959 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308ba30-3954-4f04-9c28-e0dd587b15ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:05.021298 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308ba30-3954-4f04-9c28-e0dd587b15ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:05.022652 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308ba30-3954-4f04-9c28-e0dd587b15ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:05.030951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3308ba30-3954-4f04-9c28-e0dd587b15ec admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:05.054985 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1138/2303] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:08:05.060714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac7f4c-57ad-436c-84f1-d347c87a8670 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:05.097031 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac7f4c-57ad-436c-84f1-d347c87a8670 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:05.109239 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c51166-91eb-41ad-8b50-07e69819c2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:05.110502 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c51166-91eb-41ad-8b50-07e69819c2c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:05.111022 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c51166-91eb-41ad-8b50-07e69819c2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:05.111503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c51166-91eb-41ad-8b50-07e69819c2c1 admin admin] PATH_INFO: `/v3/projects/f20903716e7e4b1fa138d8ddc6e1bdf5/users/fddd397977d34ae39b607dede45855bc/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:05.139668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c51166-91eb-41ad-8b50-07e69819c2c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f20903716e7e4b1fa138d8ddc6e1bdf5, user_id=fddd397977d34ae39b607dede45855bc, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:05.152838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c51166-91eb-41ad-8b50-07e69819c2c1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:05.197959 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1166/2304] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:08:05 2022] PUT /identity/v3/projects/f20903716e7e4b1fa138d8ddc6e1bdf5/users/fddd397977d34ae39b607dede45855bc/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:05.208578 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782994c-368a-470a-aa12-a0c1d7916c4b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:05.209156 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782994c-368a-470a-aa12-a0c1d7916c4b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:05.209423 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782994c-368a-470a-aa12-a0c1d7916c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:05.248084 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a782994c-368a-470a-aa12-a0c1d7916c4b None None] MFA Rules not processed for user `fddd397977d34ae39b607dede45855bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:05.249397 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a782994c-368a-470a-aa12-a0c1d7916c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:05.342862 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1139/2305] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:05.350412 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c51166-91eb-41ad-8b50-07e69819c2c1 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:05.354368 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-32c51166-91eb-41ad-8b50-07e69819c2c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:05.507156 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c51166-91eb-41ad-8b50-07e69819c2c1 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:05.513376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55372d7-8cca-4609-9251-3c620821f4dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:05.515187 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55372d7-8cca-4609-9251-3c620821f4dc service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:05.515626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55372d7-8cca-4609-9251-3c620821f4dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:05.516338 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55372d7-8cca-4609-9251-3c620821f4dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:05.529826 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55372d7-8cca-4609-9251-3c620821f4dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:05.531571 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55372d7-8cca-4609-9251-3c620821f4dc service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:05.543266 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1167/2306] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:08:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:08:06.454896 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884debf9-445c-4539-bf38-1f4bafc4c4c0 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:06.456020 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884debf9-445c-4539-bf38-1f4bafc4c4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:06.456610 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884debf9-445c-4539-bf38-1f4bafc4c4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:06.481104 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-884debf9-445c-4539-bf38-1f4bafc4c4c0 None None] MFA Rules not processed for user `19140330ae664e45b85f47f321496191`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:06.483778 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-884debf9-445c-4539-bf38-1f4bafc4c4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:06.572543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1140/2307] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:08.342214 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d51db-66ba-4ce5-a251-8dde2856b137 None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:08.343592 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d51db-66ba-4ce5-a251-8dde2856b137 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:08.344588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d51db-66ba-4ce5-a251-8dde2856b137 None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:08.346003 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1168/2308] 10.222.0.13 () {54 vars in 921 bytes} [Sat Jun 25 16:08:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:08:08.357856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611d022-8633-4d15-8ddb-d475613f5da8 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:08.358430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611d022-8633-4d15-8ddb-d475613f5da8 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:08.358430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611d022-8633-4d15-8ddb-d475613f5da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:08.367215 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:08.381150 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5611d022-8633-4d15-8ddb-d475613f5da8 None None] MFA Rules not processed for user `474c468a610140f8a5b18d5a1ed1d18e`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:08.382648 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5611d022-8633-4d15-8ddb-d475613f5da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:08.469690 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1141/2309] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Jun 25 16:08:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:08.482035 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d51db-66ba-4ce5-a251-8dde2856b137 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:08.484490 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-071d51db-66ba-4ce5-a251-8dde2856b137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:08.511356 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d51db-66ba-4ce5-a251-8dde2856b137 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:08.520435 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631bd5cf-cffa-4576-8a57-32a70885aae5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474c468a610140f8a5b18d5a1ed1d18e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:08.522724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631bd5cf-cffa-4576-8a57-32a70885aae5 service placement] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:08.523434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631bd5cf-cffa-4576-8a57-32a70885aae5 service placement] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:08.524162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631bd5cf-cffa-4576-8a57-32a70885aae5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:08.538420 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631bd5cf-cffa-4576-8a57-32a70885aae5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:08.541049 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631bd5cf-cffa-4576-8a57-32a70885aae5 service placement] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:08.554464 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1169/2310] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:08:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:08:09.136627 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611d022-8633-4d15-8ddb-d475613f5da8 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:09.138921 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5611d022-8633-4d15-8ddb-d475613f5da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:09.219650 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611d022-8633-4d15-8ddb-d475613f5da8 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:09.224957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad4806-98a7-4a8f-9fa6-23fa424ab167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:09.225836 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad4806-98a7-4a8f-9fa6-23fa424ab167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:09.226103 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad4806-98a7-4a8f-9fa6-23fa424ab167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:09.226373 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad4806-98a7-4a8f-9fa6-23fa424ab167 admin admin] PATH_INFO: `/v3/users/ba1c6c6fa9e44cf080f103bf98b04e5f` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:09.227595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad4806-98a7-4a8f-9fa6-23fa424ab167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba1c6c6fa9e44cf080f103bf98b04e5f)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:09.229686 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad4806-98a7-4a8f-9fa6-23fa424ab167 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:09.240966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631bd5cf-cffa-4576-8a57-32a70885aae5 service placement] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:09.252854 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631bd5cf-cffa-4576-8a57-32a70885aae5 service placement] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:09.259573 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b86d1f-cd9a-4775-8b9e-5680fbda5627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:09.260506 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b86d1f-cd9a-4775-8b9e-5680fbda5627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:09.260943 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b86d1f-cd9a-4775-8b9e-5680fbda5627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:09.261226 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b86d1f-cd9a-4775-8b9e-5680fbda5627 admin admin] PATH_INFO: `/v3/users/fddd397977d34ae39b607dede45855bc` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:09.262775 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b86d1f-cd9a-4775-8b9e-5680fbda5627 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fddd397977d34ae39b607dede45855bc)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:09.265228 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b86d1f-cd9a-4775-8b9e-5680fbda5627 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:09.301356 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a7ad4806-98a7-4a8f-9fa6-23fa424ab167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba1c6c6fa9e44cf080f103bf98b04e5f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:09.302099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a7ad4806-98a7-4a8f-9fa6-23fa424ab167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba1c6c6fa9e44cf080f103bf98b04e5f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:09.312907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a7ad4806-98a7-4a8f-9fa6-23fa424ab167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba1c6c6fa9e44cf080f103bf98b04e5f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:09.327944 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a7ad4806-98a7-4a8f-9fa6-23fa424ab167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba1c6c6fa9e44cf080f103bf98b04e5f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:09.328951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-a7ad4806-98a7-4a8f-9fa6-23fa424ab167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba1c6c6fa9e44cf080f103bf98b04e5f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:09.339781 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e8b86d1f-cd9a-4775-8b9e-5680fbda5627 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fddd397977d34ae39b607dede45855bc'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:09.350372 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e8b86d1f-cd9a-4775-8b9e-5680fbda5627 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fddd397977d34ae39b607dede45855bc'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:09.351147 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e8b86d1f-cd9a-4775-8b9e-5680fbda5627 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fddd397977d34ae39b607dede45855bc'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:09.354791 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1142/2311] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:08:09 2022] DELETE /identity/v3/users/ba1c6c6fa9e44cf080f103bf98b04e5f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:09.372763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e8b86d1f-cd9a-4775-8b9e-5680fbda5627 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fddd397977d34ae39b607dede45855bc'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:09.375054 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e8b86d1f-cd9a-4775-8b9e-5680fbda5627 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fddd397977d34ae39b607dede45855bc'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:09.419020 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1170/2312] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:08:09 2022] DELETE /identity/v3/users/fddd397977d34ae39b607dede45855bc => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:09.423567 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebeba2e-dca8-46cd-8a0f-47481f67c741 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:09.424537 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebeba2e-dca8-46cd-8a0f-47481f67c741 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:09.424922 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebeba2e-dca8-46cd-8a0f-47481f67c741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:09.461622 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-8ebeba2e-dca8-46cd-8a0f-47481f67c741 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:09.463208 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8ebeba2e-dca8-46cd-8a0f-47481f67c741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:09.486845 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:09.550550 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1143/2313] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:09.576125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b86d1f-cd9a-4775-8b9e-5680fbda5627 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:09.577595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e8b86d1f-cd9a-4775-8b9e-5680fbda5627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:09.594453 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:09.608075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebeba2e-dca8-46cd-8a0f-47481f67c741 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:09.610866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8ebeba2e-dca8-46cd-8a0f-47481f67c741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:09.626343 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebeba2e-dca8-46cd-8a0f-47481f67c741 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:09.631443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265295d-a108-4d84-85ac-706590c83fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:09.632929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265295d-a108-4d84-85ac-706590c83fdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:09.633312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265295d-a108-4d84-85ac-706590c83fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:09.633686 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265295d-a108-4d84-85ac-706590c83fdc admin admin] PATH_INFO: `/v3/projects/2245fe1962f2499db7c429815fd4d8ef` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:09.639199 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265295d-a108-4d84-85ac-706590c83fdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2245fe1962f2499db7c429815fd4d8ef)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:09.650955 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265295d-a108-4d84-85ac-706590c83fdc admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:09.745135 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b86d1f-cd9a-4775-8b9e-5680fbda5627 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:09.797870 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-2265295d-a108-4d84-85ac-706590c83fdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2245fe1962f2499db7c429815fd4d8ef', 'request_id': 'req-2265295d-a108-4d84-85ac-706590c83fdc'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:09.810977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-2265295d-a108-4d84-85ac-706590c83fdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2245fe1962f2499db7c429815fd4d8ef', 'request_id': 'req-2265295d-a108-4d84-85ac-706590c83fdc'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:09.835127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-2265295d-a108-4d84-85ac-706590c83fdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2245fe1962f2499db7c429815fd4d8ef', 'request_id': 'req-2265295d-a108-4d84-85ac-706590c83fdc'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:09.863787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad343b5-a19d-43bc-a0fa-9694ea80d6c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:09.863787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad343b5-a19d-43bc-a0fa-9694ea80d6c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:09.863787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad343b5-a19d-43bc-a0fa-9694ea80d6c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:09.863787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad343b5-a19d-43bc-a0fa-9694ea80d6c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:09.878821 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-2265295d-a108-4d84-85ac-706590c83fdc admin admin] The token cache is being invalidate because project 2245fe1962f2499db7c429815fd4d8ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:08:09.879100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-2265295d-a108-4d84-85ac-706590c83fdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:09.879827 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-2265295d-a108-4d84-85ac-706590c83fdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:09.882412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1144/2314] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:08:09 2022] DELETE /identity/v3/projects/2245fe1962f2499db7c429815fd4d8ef => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:09.888963 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265295d-a108-4d84-85ac-706590c83fdc admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:09.892678 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2265295d-a108-4d84-85ac-706590c83fdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:10.040425 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad343b5-a19d-43bc-a0fa-9694ea80d6c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:10.042358 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad343b5-a19d-43bc-a0fa-9694ea80d6c2 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:10.046912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265295d-a108-4d84-85ac-706590c83fdc admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:10.052040 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9025c5-d829-4136-b120-4ce5c995eac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:10.053177 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9025c5-d829-4136-b120-4ce5c995eac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:10.053461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9025c5-d829-4136-b120-4ce5c995eac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:10.053844 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9025c5-d829-4136-b120-4ce5c995eac9 admin admin] PATH_INFO: `/v3/users/bddfa4e17aaa436cb32201587d0ce2b5` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:10.055338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9025c5-d829-4136-b120-4ce5c995eac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bddfa4e17aaa436cb32201587d0ce2b5)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:10.060212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1171/2315] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:08:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:08:10.061267 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9025c5-d829-4136-b120-4ce5c995eac9 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:10.156715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:10.159149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1b9025c5-d829-4136-b120-4ce5c995eac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bddfa4e17aaa436cb32201587d0ce2b5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:10.160274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1b9025c5-d829-4136-b120-4ce5c995eac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bddfa4e17aaa436cb32201587d0ce2b5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:10.177582 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1b9025c5-d829-4136-b120-4ce5c995eac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bddfa4e17aaa436cb32201587d0ce2b5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:10.186656 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1b9025c5-d829-4136-b120-4ce5c995eac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bddfa4e17aaa436cb32201587d0ce2b5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:10.187531 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-1b9025c5-d829-4136-b120-4ce5c995eac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bddfa4e17aaa436cb32201587d0ce2b5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:10.254942 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1145/2316] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:08:09 2022] DELETE /identity/v3/users/bddfa4e17aaa436cb32201587d0ce2b5 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:10.681318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad343b5-a19d-43bc-a0fa-9694ea80d6c2 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:10.702062 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad343b5-a19d-43bc-a0fa-9694ea80d6c2 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:10.828179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fdbd6-5df3-4737-a81f-7ef1668509e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:10.829303 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fdbd6-5df3-4737-a81f-7ef1668509e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:10.829586 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fdbd6-5df3-4737-a81f-7ef1668509e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:10.829858 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fdbd6-5df3-4737-a81f-7ef1668509e1 admin admin] PATH_INFO: `/v3/projects/126479dc7bc54e28906eedd13c141edf` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:10.831162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988fdbd6-5df3-4737-a81f-7ef1668509e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=126479dc7bc54e28906eedd13c141edf)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:10.833280 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988fdbd6-5df3-4737-a81f-7ef1668509e1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:10.887544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:10.925312 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-988fdbd6-5df3-4737-a81f-7ef1668509e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '126479dc7bc54e28906eedd13c141edf', 'request_id': 'req-988fdbd6-5df3-4737-a81f-7ef1668509e1'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:10.938499 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-988fdbd6-5df3-4737-a81f-7ef1668509e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '126479dc7bc54e28906eedd13c141edf', 'request_id': 'req-988fdbd6-5df3-4737-a81f-7ef1668509e1'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:10.945926 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-988fdbd6-5df3-4737-a81f-7ef1668509e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '126479dc7bc54e28906eedd13c141edf', 'request_id': 'req-988fdbd6-5df3-4737-a81f-7ef1668509e1'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:10.971660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-988fdbd6-5df3-4737-a81f-7ef1668509e1 admin admin] The token cache is being invalidate because project 126479dc7bc54e28906eedd13c141edf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:08:10.972203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-988fdbd6-5df3-4737-a81f-7ef1668509e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:10.973378 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-988fdbd6-5df3-4737-a81f-7ef1668509e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:10.975448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1172/2317] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:08:10 2022] DELETE /identity/v3/projects/126479dc7bc54e28906eedd13c141edf => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:10.980579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2a3e6-688a-48e1-b4c3-04e0a742efb7 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:10.981148 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2a3e6-688a-48e1-b4c3-04e0a742efb7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:10.981524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2a3e6-688a-48e1-b4c3-04e0a742efb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:11.033736 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-dcc2a3e6-688a-48e1-b4c3-04e0a742efb7 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:11.035369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-dcc2a3e6-688a-48e1-b4c3-04e0a742efb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:11.067370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fdbd6-5df3-4737-a81f-7ef1668509e1 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:11.069059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-988fdbd6-5df3-4737-a81f-7ef1668509e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:11.150753 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fdbd6-5df3-4737-a81f-7ef1668509e1 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:11.155896 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fcdd3-a58b-4824-8f7a-63fb630b7715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:11.156843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fcdd3-a58b-4824-8f7a-63fb630b7715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:11.157125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fcdd3-a58b-4824-8f7a-63fb630b7715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:11.157401 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fcdd3-a58b-4824-8f7a-63fb630b7715 admin admin] PATH_INFO: `/v3/projects/f20903716e7e4b1fa138d8ddc6e1bdf5` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:11.159072 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7fcdd3-a58b-4824-8f7a-63fb630b7715 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f20903716e7e4b1fa138d8ddc6e1bdf5)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:11.161567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7fcdd3-a58b-4824-8f7a-63fb630b7715 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:11.179692 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1146/2318] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:11.187726 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdc375-d04b-4ce7-aec8-8fe07e832689 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:11.188355 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdc375-d04b-4ce7-aec8-8fe07e832689 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:11.188934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fdc375-d04b-4ce7-aec8-8fe07e832689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:11.249858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a9fdc375-d04b-4ce7-aec8-8fe07e832689 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:11.251453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a9fdc375-d04b-4ce7-aec8-8fe07e832689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:11.276812 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1147/2319] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:11.280929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fdc375-d04b-4ce7-aec8-8fe07e832689 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:11.330207 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ca7fcdd3-a58b-4824-8f7a-63fb630b7715 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f20903716e7e4b1fa138d8ddc6e1bdf5', 'request_id': 'req-ca7fcdd3-a58b-4824-8f7a-63fb630b7715'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:11.337698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fdc375-d04b-4ce7-aec8-8fe07e832689 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:11.349130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ca7fcdd3-a58b-4824-8f7a-63fb630b7715 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f20903716e7e4b1fa138d8ddc6e1bdf5', 'request_id': 'req-ca7fcdd3-a58b-4824-8f7a-63fb630b7715'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:11.366481 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ca7fcdd3-a58b-4824-8f7a-63fb630b7715 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f20903716e7e4b1fa138d8ddc6e1bdf5', 'request_id': 'req-ca7fcdd3-a58b-4824-8f7a-63fb630b7715'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:11.398896 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ca7fcdd3-a58b-4824-8f7a-63fb630b7715 admin admin] The token cache is being invalidate because project f20903716e7e4b1fa138d8ddc6e1bdf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:08:11.399112 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ca7fcdd3-a58b-4824-8f7a-63fb630b7715 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:11.399873 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-ca7fcdd3-a58b-4824-8f7a-63fb630b7715 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:11.401849 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1173/2320] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:08:11 2022] DELETE /identity/v3/projects/f20903716e7e4b1fa138d8ddc6e1bdf5 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:11.416480 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21c4ec-6928-4c7c-b70a-f1ad19718727 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:11.416777 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21c4ec-6928-4c7c-b70a-f1ad19718727 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:11.417200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21c4ec-6928-4c7c-b70a-f1ad19718727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:11.456852 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5c21c4ec-6928-4c7c-b70a-f1ad19718727 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:11.458198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5c21c4ec-6928-4c7c-b70a-f1ad19718727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:11.466868 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d3587-0375-4168-863b-115d13f3319e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:11.467150 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d3587-0375-4168-863b-115d13f3319e admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:11.467732 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d3587-0375-4168-863b-115d13f3319e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:11.468144 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d3587-0375-4168-863b-115d13f3319e admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:11.469060 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d3587-0375-4168-863b-115d13f3319e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:08:11.469570 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d3587-0375-4168-863b-115d13f3319e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:11.472513 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d3587-0375-4168-863b-115d13f3319e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:11.488316 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1148/2321] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:08:11.493403 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d3587-0375-4168-863b-115d13f3319e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:11.495228 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-097d3587-0375-4168-863b-115d13f3319e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:11.505545 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:11.511771 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d3587-0375-4168-863b-115d13f3319e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:11.522761 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e2cf3-6325-41a9-b028-3708c0bd8fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:11.524224 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e2cf3-6325-41a9-b028-3708c0bd8fa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:11.524795 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e2cf3-6325-41a9-b028-3708c0bd8fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:11.525133 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e2cf3-6325-41a9-b028-3708c0bd8fa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:11.526167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e2cf3-6325-41a9-b028-3708c0bd8fa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:11.528755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e2cf3-6325-41a9-b028-3708c0bd8fa2 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:11.541014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1174/2322] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:11.548616 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645209e1-bf9d-4e1e-8e0a-911e9b47e0af None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:11.549189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645209e1-bf9d-4e1e-8e0a-911e9b47e0af None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:11.549314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645209e1-bf9d-4e1e-8e0a-911e9b47e0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:11.570091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-645209e1-bf9d-4e1e-8e0a-911e9b47e0af None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:11.571485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-645209e1-bf9d-4e1e-8e0a-911e9b47e0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:11.581572 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1175/2323] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:08:11.586574 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645209e1-bf9d-4e1e-8e0a-911e9b47e0af None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:11.594058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1149/2324] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:08:11 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:08:11.595070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e2cf3-6325-41a9-b028-3708c0bd8fa2 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:11.597765 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645209e1-bf9d-4e1e-8e0a-911e9b47e0af None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:11.666982 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:11.686132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bffadf-7ba0-40e4-b2bd-ba600fc1a7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:11.686866 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bffadf-7ba0-40e4-b2bd-ba600fc1a7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:11.687267 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bffadf-7ba0-40e4-b2bd-ba600fc1a7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:11.687604 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bffadf-7ba0-40e4-b2bd-ba600fc1a7e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:11.688518 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bffadf-7ba0-40e4-b2bd-ba600fc1a7e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:08:11.688894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bffadf-7ba0-40e4-b2bd-ba600fc1a7e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:11.691665 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bffadf-7ba0-40e4-b2bd-ba600fc1a7e9 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:11.705988 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1176/2325] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:08:11.710565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bffadf-7ba0-40e4-b2bd-ba600fc1a7e9 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:11.726120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bffadf-7ba0-40e4-b2bd-ba600fc1a7e9 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:11.733971 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a75d2c-c96d-45f2-94c0-d2db65130b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:11.734870 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a75d2c-c96d-45f2-94c0-d2db65130b54 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:11.735039 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a75d2c-c96d-45f2-94c0-d2db65130b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:11.735429 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a75d2c-c96d-45f2-94c0-d2db65130b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:11.736397 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a75d2c-c96d-45f2-94c0-d2db65130b54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:11.738902 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a75d2c-c96d-45f2-94c0-d2db65130b54 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:11.739811 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e2cf3-6325-41a9-b028-3708c0bd8fa2 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:11.746956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516267e-dd2f-415e-a80e-bd1108a144ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:11.747647 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516267e-dd2f-415e-a80e-bd1108a144ab admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:11.747926 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516267e-dd2f-415e-a80e-bd1108a144ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:11.748197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516267e-dd2f-415e-a80e-bd1108a144ab admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:11.748924 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3516267e-dd2f-415e-a80e-bd1108a144ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:11.750885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3516267e-dd2f-415e-a80e-bd1108a144ab admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:11.805363 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1150/2327] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:08:11 2022] POST /identity/v3/users => generated 292 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:08:11.806963 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516267e-dd2f-415e-a80e-bd1108a144ab admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:11.807484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1177/2326] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:08:11 2022] POST /identity/v3/projects => generated 377 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:08:11.808401 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a75d2c-c96d-45f2-94c0-d2db65130b54 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:11.822338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516267e-dd2f-415e-a80e-bd1108a144ab admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:11.953898 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a605358-fb12-4153-b3e0-97de9a1176c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:11.954546 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a605358-fb12-4153-b3e0-97de9a1176c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:11.954828 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a605358-fb12-4153-b3e0-97de9a1176c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:11.955101 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a605358-fb12-4153-b3e0-97de9a1176c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:11.955854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a605358-fb12-4153-b3e0-97de9a1176c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:11.957773 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a605358-fb12-4153-b3e0-97de9a1176c3 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:11.969745 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a75d2c-c96d-45f2-94c0-d2db65130b54 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:11.973144 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1151/2328] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:08:11.977542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25267c-39b0-47dc-b2c4-a6c1265bfae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:11.978772 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25267c-39b0-47dc-b2c4-a6c1265bfae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:11.979226 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a605358-fb12-4153-b3e0-97de9a1176c3 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:11.979952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25267c-39b0-47dc-b2c4-a6c1265bfae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:11.980742 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25267c-39b0-47dc-b2c4-a6c1265bfae9 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:11.983199 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25267c-39b0-47dc-b2c4-a6c1265bfae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:11.994125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25267c-39b0-47dc-b2c4-a6c1265bfae9 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:12.005259 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a605358-fb12-4153-b3e0-97de9a1176c3 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:12.016694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864214fb-16b3-49d3-854c-19050490522d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:12.017524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864214fb-16b3-49d3-854c-19050490522d admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:12.018056 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864214fb-16b3-49d3-854c-19050490522d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:12.018423 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864214fb-16b3-49d3-854c-19050490522d admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:12.019283 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864214fb-16b3-49d3-854c-19050490522d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:12.026541 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864214fb-16b3-49d3-854c-19050490522d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:12.030008 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1178/2329] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:08:11 2022] POST /identity/v3/users => generated 289 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:08:12.034761 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25267c-39b0-47dc-b2c4-a6c1265bfae9 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:12.038603 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1152/2330] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:08:12.043930 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864214fb-16b3-49d3-854c-19050490522d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:12.054570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25267c-39b0-47dc-b2c4-a6c1265bfae9 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:12.064906 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864214fb-16b3-49d3-854c-19050490522d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:12.074815 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975efc71-3885-4058-af05-b8000766af34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:12.075092 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d8c4f7-c4b0-446d-880d-8cd7b8fdb73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:12.076161 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975efc71-3885-4058-af05-b8000766af34 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:12.076300 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8c4f7-c4b0-446d-880d-8cd7b8fdb73b admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:12.076733 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8c4f7-c4b0-446d-880d-8cd7b8fdb73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:12.076869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975efc71-3885-4058-af05-b8000766af34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:12.077482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975efc71-3885-4058-af05-b8000766af34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:12.077789 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8c4f7-c4b0-446d-880d-8cd7b8fdb73b admin admin] PATH_INFO: `/v3/projects/a438bb22ca6c456f894da8d463d5d45c/users/78fc889b85d84830971575a60b2ef710/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:12.079107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975efc71-3885-4058-af05-b8000766af34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:12.088923 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975efc71-3885-4058-af05-b8000766af34 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:12.098267 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d8c4f7-c4b0-446d-880d-8cd7b8fdb73b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a438bb22ca6c456f894da8d463d5d45c, user_id=78fc889b85d84830971575a60b2ef710, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:12.101463 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d8c4f7-c4b0-446d-880d-8cd7b8fdb73b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:12.109956 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1179/2331] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:08:12.111153 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975efc71-3885-4058-af05-b8000766af34 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:12.120569 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1153/2332] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:08:12 2022] PUT /identity/v3/projects/a438bb22ca6c456f894da8d463d5d45c/users/78fc889b85d84830971575a60b2ef710/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:12.134026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cd84c-4081-4a43-a2fa-eb1c5c4b82eb None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:12.135375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cd84c-4081-4a43-a2fa-eb1c5c4b82eb None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:12.135752 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cd84c-4081-4a43-a2fa-eb1c5c4b82eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:12.141559 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975efc71-3885-4058-af05-b8000766af34 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:12.185952 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:12.188566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-346cd84c-4081-4a43-a2fa-eb1c5c4b82eb None None] MFA Rules not processed for user `78fc889b85d84830971575a60b2ef710`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:12.190128 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-346cd84c-4081-4a43-a2fa-eb1c5c4b82eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:12.256219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5311c06-f607-4b09-a9bc-343c6f629a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:12.257460 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5311c06-f607-4b09-a9bc-343c6f629a18 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:12.257846 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5311c06-f607-4b09-a9bc-343c6f629a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:12.258217 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5311c06-f607-4b09-a9bc-343c6f629a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:12.259070 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5311c06-f607-4b09-a9bc-343c6f629a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:12.261469 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5311c06-f607-4b09-a9bc-343c6f629a18 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:12.278469 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1180/2333] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:08:12.279811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5311c06-f607-4b09-a9bc-343c6f629a18 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:12.291446 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5311c06-f607-4b09-a9bc-343c6f629a18 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:12.299326 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20ae58-a760-4f5d-81b8-06a3365229e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:12.300029 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20ae58-a760-4f5d-81b8-06a3365229e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:12.300434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20ae58-a760-4f5d-81b8-06a3365229e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:12.300816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20ae58-a760-4f5d-81b8-06a3365229e8 admin admin] PATH_INFO: `/v3/projects/065373b938b744eeb805f8878e83f716/users/c6fc8b2e9c1e49848d9a6536e1e8a30f/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:12.315417 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1154/2334] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:12.319797 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20ae58-a760-4f5d-81b8-06a3365229e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=065373b938b744eeb805f8878e83f716, user_id=c6fc8b2e9c1e49848d9a6536e1e8a30f, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:12.322757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20ae58-a760-4f5d-81b8-06a3365229e8 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:12.325068 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cd84c-4081-4a43-a2fa-eb1c5c4b82eb None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:12.326653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-346cd84c-4081-4a43-a2fa-eb1c5c4b82eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:12.339484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:12.365454 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1181/2335] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:08:12 2022] PUT /identity/v3/projects/065373b938b744eeb805f8878e83f716/users/c6fc8b2e9c1e49848d9a6536e1e8a30f/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:12.376440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddeff3-203e-4577-812a-0e585f8dab19 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:12.378444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddeff3-203e-4577-812a-0e585f8dab19 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:12.381458 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddeff3-203e-4577-812a-0e585f8dab19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:12.447396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-72ddeff3-203e-4577-812a-0e585f8dab19 None None] MFA Rules not processed for user `c6fc8b2e9c1e49848d9a6536e1e8a30f`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:12.448801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-72ddeff3-203e-4577-812a-0e585f8dab19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:12.471282 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cd84c-4081-4a43-a2fa-eb1c5c4b82eb None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:12.534578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1182/2336] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:08:12.539760 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddeff3-203e-4577-812a-0e585f8dab19 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:12.552340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417911d-a0ac-44b1-95dc-e0ea8624f99d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:12.552955 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417911d-a0ac-44b1-95dc-e0ea8624f99d service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:12.553396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417911d-a0ac-44b1-95dc-e0ea8624f99d service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:12.553768 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417911d-a0ac-44b1-95dc-e0ea8624f99d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:12.641931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddeff3-203e-4577-812a-0e585f8dab19 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:12.650552 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a94321-d92a-4a71-99d6-077d1df722e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:12.651327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a94321-d92a-4a71-99d6-077d1df722e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:12.651658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a94321-d92a-4a71-99d6-077d1df722e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:12.651981 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a94321-d92a-4a71-99d6-077d1df722e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:12.652728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a94321-d92a-4a71-99d6-077d1df722e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:12.655889 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a94321-d92a-4a71-99d6-077d1df722e9 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:12.658780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a417911d-a0ac-44b1-95dc-e0ea8624f99d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:12.660334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a417911d-a0ac-44b1-95dc-e0ea8624f99d service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:12.694795 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1155/2337] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:08:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:08:12.716303 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1183/2338] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:08:12 2022] POST /identity/v3/projects => generated 375 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:08:12.718696 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417911d-a0ac-44b1-95dc-e0ea8624f99d service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:12.721096 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a94321-d92a-4a71-99d6-077d1df722e9 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:12.741739 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:12.745774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417911d-a0ac-44b1-95dc-e0ea8624f99d service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:12.872142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a94321-d92a-4a71-99d6-077d1df722e9 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:12.878981 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8f69a-974d-4bee-aff4-39f52d843fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:12.879923 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8f69a-974d-4bee-aff4-39f52d843fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:12.880201 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8f69a-974d-4bee-aff4-39f52d843fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:12.880508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8f69a-974d-4bee-aff4-39f52d843fb1 admin admin] PATH_INFO: `/v3/users/78fc889b85d84830971575a60b2ef710` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:12.881736 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8f69a-974d-4bee-aff4-39f52d843fb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78fc889b85d84830971575a60b2ef710)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:12.883676 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8f69a-974d-4bee-aff4-39f52d843fb1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:12.884097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a40d2d-097f-4b63-a57f-6f01c16b901f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:12.885391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a40d2d-097f-4b63-a57f-6f01c16b901f admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:12.886152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a40d2d-097f-4b63-a57f-6f01c16b901f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:12.886971 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a40d2d-097f-4b63-a57f-6f01c16b901f admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:12.888286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a40d2d-097f-4b63-a57f-6f01c16b901f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:12.892378 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a40d2d-097f-4b63-a57f-6f01c16b901f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:12.923386 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:12.945530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1184/2339] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:08:12 2022] POST /identity/v3/users => generated 288 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:08:12.951199 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a40d2d-097f-4b63-a57f-6f01c16b901f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:12.968170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a40d2d-097f-4b63-a57f-6f01c16b901f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:12.971144 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-14a8f69a-974d-4bee-aff4-39f52d843fb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78fc889b85d84830971575a60b2ef710'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:12.971867 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-14a8f69a-974d-4bee-aff4-39f52d843fb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78fc889b85d84830971575a60b2ef710'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:12.976623 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c89ab-0d5c-4efa-a6df-c546e70e2640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:12.978295 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c89ab-0d5c-4efa-a6df-c546e70e2640 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:12.978674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c89ab-0d5c-4efa-a6df-c546e70e2640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:12.978945 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c89ab-0d5c-4efa-a6df-c546e70e2640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:12.979884 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c89ab-0d5c-4efa-a6df-c546e70e2640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:12.982391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-14a8f69a-974d-4bee-aff4-39f52d843fb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78fc889b85d84830971575a60b2ef710'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:12.985051 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c89ab-0d5c-4efa-a6df-c546e70e2640 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:13.000289 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-14a8f69a-974d-4bee-aff4-39f52d843fb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78fc889b85d84830971575a60b2ef710'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:13.001539 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-14a8f69a-974d-4bee-aff4-39f52d843fb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78fc889b85d84830971575a60b2ef710'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:13.005557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1185/2340] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:08:13.012567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c89ab-0d5c-4efa-a6df-c546e70e2640 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:13.015530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-612c89ab-0d5c-4efa-a6df-c546e70e2640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:13.025511 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1156/2341] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:08:12 2022] DELETE /identity/v3/users/78fc889b85d84830971575a60b2ef710 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:13.030694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef2d88-33c6-4a47-abc6-267247648342 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:13.031336 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef2d88-33c6-4a47-abc6-267247648342 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:13.031919 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef2d88-33c6-4a47-abc6-267247648342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:13.035926 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c89ab-0d5c-4efa-a6df-c546e70e2640 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:13.058141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-09ef2d88-33c6-4a47-abc6-267247648342 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:13.059452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-09ef2d88-33c6-4a47-abc6-267247648342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:13.130976 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d190b06-de9d-488c-b1a2-43701f4b4fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:13.131570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d190b06-de9d-488c-b1a2-43701f4b4fda admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:13.131941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d190b06-de9d-488c-b1a2-43701f4b4fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:13.132313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d190b06-de9d-488c-b1a2-43701f4b4fda admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:13.133143 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d190b06-de9d-488c-b1a2-43701f4b4fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:13.134897 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d190b06-de9d-488c-b1a2-43701f4b4fda admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:13.151356 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1186/2342] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:08:13.155704 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d190b06-de9d-488c-b1a2-43701f4b4fda admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:13.167191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d190b06-de9d-488c-b1a2-43701f4b4fda admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:13.172544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a135cc-ce41-48c4-91cc-39e01454256a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:13.174464 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a135cc-ce41-48c4-91cc-39e01454256a admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:13.174836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a135cc-ce41-48c4-91cc-39e01454256a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:13.175143 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a135cc-ce41-48c4-91cc-39e01454256a admin admin] PATH_INFO: `/v3/projects/8d582381453440fda980dd7356d1daff/users/3a5539386ed24df0ae87b782c376b76c/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:13.208980 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1157/2343] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:13.210357 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a135cc-ce41-48c4-91cc-39e01454256a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d582381453440fda980dd7356d1daff, user_id=3a5539386ed24df0ae87b782c376b76c, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:13.215212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a135cc-ce41-48c4-91cc-39e01454256a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:13.231657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1187/2344] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:08:13 2022] PUT /identity/v3/projects/8d582381453440fda980dd7356d1daff/users/3a5539386ed24df0ae87b782c376b76c/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:13.242079 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca28b5-27f7-4c94-9f70-35530e9c52d8 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:13.242360 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca28b5-27f7-4c94-9f70-35530e9c52d8 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:13.242632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca28b5-27f7-4c94-9f70-35530e9c52d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:13.272252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a135cc-ce41-48c4-91cc-39e01454256a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:13.274013 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e6a135cc-ce41-48c4-91cc-39e01454256a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:13.302157 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-7aca28b5-27f7-4c94-9f70-35530e9c52d8 None None] MFA Rules not processed for user `3a5539386ed24df0ae87b782c376b76c`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:13.303410 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7aca28b5-27f7-4c94-9f70-35530e9c52d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:13.407936 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a135cc-ce41-48c4-91cc-39e01454256a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:13.413736 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1a644-3a90-47b7-b7bf-3f369e2a660f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:13.414843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1a644-3a90-47b7-b7bf-3f369e2a660f service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:13.415556 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1a644-3a90-47b7-b7bf-3f369e2a660f service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:13.416039 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1a644-3a90-47b7-b7bf-3f369e2a660f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:13.440317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:13.474568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1158/2345] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:08:13.479479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca28b5-27f7-4c94-9f70-35530e9c52d8 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:13.578756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:13.612495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1a644-3a90-47b7-b7bf-3f369e2a660f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:13.614783 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1a644-3a90-47b7-b7bf-3f369e2a660f service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:13.650023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1188/2346] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:08:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:08:13.693084 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca28b5-27f7-4c94-9f70-35530e9c52d8 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:13.701236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b5ea5-12e4-48ae-a63b-8a6138ea20f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:13.702420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b5ea5-12e4-48ae-a63b-8a6138ea20f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:13.702910 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b5ea5-12e4-48ae-a63b-8a6138ea20f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:13.703462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b5ea5-12e4-48ae-a63b-8a6138ea20f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:13.704372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b5ea5-12e4-48ae-a63b-8a6138ea20f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:13.716164 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8b64a-9848-4a41-8063-b6e75a2e140a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:13.716484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8b64a-9848-4a41-8063-b6e75a2e140a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:13.717028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8b64a-9848-4a41-8063-b6e75a2e140a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:13.721039 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b5ea5-12e4-48ae-a63b-8a6138ea20f8 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:13.769706 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-f6e8b64a-9848-4a41-8063-b6e75a2e140a None None] MFA Rules not processed for user `8a3137566b004edb909d4e3c702a7f59`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:13.771168 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f6e8b64a-9848-4a41-8063-b6e75a2e140a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:13.775093 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1159/2347] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:08:13 2022] POST /identity/v3/projects => generated 377 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:08:13.778984 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b5ea5-12e4-48ae-a63b-8a6138ea20f8 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:13.950692 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1189/2348] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:08:13.987666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b5ea5-12e4-48ae-a63b-8a6138ea20f8 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:14.001140 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99608a32-4e48-4829-893a-c4ea39dfbe96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:14.001995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99608a32-4e48-4829-893a-c4ea39dfbe96 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:14.002426 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99608a32-4e48-4829-893a-c4ea39dfbe96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:14.002796 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99608a32-4e48-4829-893a-c4ea39dfbe96 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:14.003657 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99608a32-4e48-4829-893a-c4ea39dfbe96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:14.016733 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99608a32-4e48-4829-893a-c4ea39dfbe96 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:14.060221 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1160/2349] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:08:13 2022] POST /identity/v3/users => generated 295 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:08:14.065152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8b64a-9848-4a41-8063-b6e75a2e140a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:14.097270 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8b64a-9848-4a41-8063-b6e75a2e140a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:14.117880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecca443-798b-4bdb-a1b9-ceabbe142af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:14.123472 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecca443-798b-4bdb-a1b9-ceabbe142af7 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:14.123472 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecca443-798b-4bdb-a1b9-ceabbe142af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:14.123472 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecca443-798b-4bdb-a1b9-ceabbe142af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:14.123472 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecca443-798b-4bdb-a1b9-ceabbe142af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:14.124528 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecca443-798b-4bdb-a1b9-ceabbe142af7 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:14.144492 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1190/2350] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:08:14.148244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99608a32-4e48-4829-893a-c4ea39dfbe96 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:14.160771 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99608a32-4e48-4829-893a-c4ea39dfbe96 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:14.166344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4b0ea-4a9c-4095-86fb-8ebb55876f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:14.167525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4b0ea-4a9c-4095-86fb-8ebb55876f3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:14.167928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4b0ea-4a9c-4095-86fb-8ebb55876f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:14.168296 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4b0ea-4a9c-4095-86fb-8ebb55876f3b admin admin] PATH_INFO: `/v3/projects/9e5f6486d9b041d2b8d0ef95c48fdba0/users/b9ccf584cbc1452ea1bc1e7b7dd33afa/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:14.189293 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4b0ea-4a9c-4095-86fb-8ebb55876f3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e5f6486d9b041d2b8d0ef95c48fdba0, user_id=b9ccf584cbc1452ea1bc1e7b7dd33afa, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:14.192471 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4b0ea-4a9c-4095-86fb-8ebb55876f3b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:14.212260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1161/2351] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:08:14 2022] PUT /identity/v3/projects/9e5f6486d9b041d2b8d0ef95c48fdba0/users/b9ccf584cbc1452ea1bc1e7b7dd33afa/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:14.216260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecca443-798b-4bdb-a1b9-ceabbe142af7 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:14.238730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecca443-798b-4bdb-a1b9-ceabbe142af7 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:14.342299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f53acf-caec-4f37-bd72-b9901bdbcc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:14.343060 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f53acf-caec-4f37-bd72-b9901bdbcc6d admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:14.343344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f53acf-caec-4f37-bd72-b9901bdbcc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:14.343621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f53acf-caec-4f37-bd72-b9901bdbcc6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:14.344892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f53acf-caec-4f37-bd72-b9901bdbcc6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:14.346333 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f53acf-caec-4f37-bd72-b9901bdbcc6d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:14.368974 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1191/2352] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:08:14.370620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4b0ea-4a9c-4095-86fb-8ebb55876f3b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:14.384635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4b0ea-4a9c-4095-86fb-8ebb55876f3b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:14.395733 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e448165-3cec-4c48-9410-6a5c0f3ee33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:14.397440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e448165-3cec-4c48-9410-6a5c0f3ee33c admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:14.400693 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e448165-3cec-4c48-9410-6a5c0f3ee33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:14.401319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e448165-3cec-4c48-9410-6a5c0f3ee33c admin admin] PATH_INFO: `/v3/projects/9e5f6486d9b041d2b8d0ef95c48fdba0/users/b9ccf584cbc1452ea1bc1e7b7dd33afa/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:14.404621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e448165-3cec-4c48-9410-6a5c0f3ee33c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e5f6486d9b041d2b8d0ef95c48fdba0, user_id=b9ccf584cbc1452ea1bc1e7b7dd33afa, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:14.407436 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e448165-3cec-4c48-9410-6a5c0f3ee33c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:14.421127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1162/2353] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:08:14 2022] PUT /identity/v3/projects/9e5f6486d9b041d2b8d0ef95c48fdba0/users/b9ccf584cbc1452ea1bc1e7b7dd33afa/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:14.431639 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cbe70-119a-43d3-aa04-c7d26fc88d6b None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:14.432357 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cbe70-119a-43d3-aa04-c7d26fc88d6b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:14.432684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cbe70-119a-43d3-aa04-c7d26fc88d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:14.470621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-773cbe70-119a-43d3-aa04-c7d26fc88d6b None None] MFA Rules not processed for user `b9ccf584cbc1452ea1bc1e7b7dd33afa`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:14.472297 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-773cbe70-119a-43d3-aa04-c7d26fc88d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:14.510983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:14.573308 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1192/2354] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:14 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:14.583101 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e448165-3cec-4c48-9410-6a5c0f3ee33c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:14.585178 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7e448165-3cec-4c48-9410-6a5c0f3ee33c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:14.688222 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e448165-3cec-4c48-9410-6a5c0f3ee33c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:14.700131 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd1ae4f-8efe-4ef3-bdde-b68884f6e85b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:14.701660 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1ae4f-8efe-4ef3-bdde-b68884f6e85b service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:14.702189 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1ae4f-8efe-4ef3-bdde-b68884f6e85b service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:14.702619 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1ae4f-8efe-4ef3-bdde-b68884f6e85b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:14.705720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-fbd1ae4f-8efe-4ef3-bdde-b68884f6e85b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:14.754645 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:14.834869 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1ae4f-8efe-4ef3-bdde-b68884f6e85b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:14.836597 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd1ae4f-8efe-4ef3-bdde-b68884f6e85b service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:14.850493 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1163/2355] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:08:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:08:14.894314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cbe70-119a-43d3-aa04-c7d26fc88d6b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:14.896541 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-773cbe70-119a-43d3-aa04-c7d26fc88d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:15.029221 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cbe70-119a-43d3-aa04-c7d26fc88d6b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:15.036046 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea70ed9-ff2d-4c2d-8416-6891d9c243cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:15.037016 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea70ed9-ff2d-4c2d-8416-6891d9c243cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:15.037309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea70ed9-ff2d-4c2d-8416-6891d9c243cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:15.037637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea70ed9-ff2d-4c2d-8416-6891d9c243cb admin admin] PATH_INFO: `/v3/projects/a438bb22ca6c456f894da8d463d5d45c` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:15.039081 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea70ed9-ff2d-4c2d-8416-6891d9c243cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a438bb22ca6c456f894da8d463d5d45c)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:15.041271 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea70ed9-ff2d-4c2d-8416-6891d9c243cb admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:15.128993 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-aea70ed9-ff2d-4c2d-8416-6891d9c243cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a438bb22ca6c456f894da8d463d5d45c', 'request_id': 'req-aea70ed9-ff2d-4c2d-8416-6891d9c243cb'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:15.134188 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-aea70ed9-ff2d-4c2d-8416-6891d9c243cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a438bb22ca6c456f894da8d463d5d45c', 'request_id': 'req-aea70ed9-ff2d-4c2d-8416-6891d9c243cb'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:15.143228 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-aea70ed9-ff2d-4c2d-8416-6891d9c243cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a438bb22ca6c456f894da8d463d5d45c', 'request_id': 'req-aea70ed9-ff2d-4c2d-8416-6891d9c243cb'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:15.161617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-aea70ed9-ff2d-4c2d-8416-6891d9c243cb admin admin] The token cache is being invalidate because project a438bb22ca6c456f894da8d463d5d45c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:08:15.163231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-aea70ed9-ff2d-4c2d-8416-6891d9c243cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:15.165405 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-aea70ed9-ff2d-4c2d-8416-6891d9c243cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:15.168238 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1193/2356] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:08:14 2022] DELETE /identity/v3/projects/a438bb22ca6c456f894da8d463d5d45c => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:15.173250 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4cc9c-051e-4dd2-9f5f-2b3a54b3787f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:15.173902 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4cc9c-051e-4dd2-9f5f-2b3a54b3787f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:15.174882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4cc9c-051e-4dd2-9f5f-2b3a54b3787f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:15.200150 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-7cc4cc9c-051e-4dd2-9f5f-2b3a54b3787f None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:15.201594 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7cc4cc9c-051e-4dd2-9f5f-2b3a54b3787f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:15.309705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1164/2357] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:08:15.318724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7747e6-01a5-4e5e-a1c0-764cf77fb6de None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:15.319434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7747e6-01a5-4e5e-a1c0-764cf77fb6de None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:15.319808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7747e6-01a5-4e5e-a1c0-764cf77fb6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:15.351144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-fd7747e6-01a5-4e5e-a1c0-764cf77fb6de None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:15.352641 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-fd7747e6-01a5-4e5e-a1c0-764cf77fb6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:15.374532 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1194/2358] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:15.379162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4cc9c-051e-4dd2-9f5f-2b3a54b3787f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:15.398402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4cc9c-051e-4dd2-9f5f-2b3a54b3787f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:15.405395 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef391b8-696b-4521-ab33-02044aca99e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:15.406203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef391b8-696b-4521-ab33-02044aca99e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:15.407709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef391b8-696b-4521-ab33-02044aca99e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:15.407709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef391b8-696b-4521-ab33-02044aca99e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:15.407709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef391b8-696b-4521-ab33-02044aca99e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:08:15.408174 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef391b8-696b-4521-ab33-02044aca99e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:15.409929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef391b8-696b-4521-ab33-02044aca99e7 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:15.425412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1165/2359] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:08:15.430333 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7747e6-01a5-4e5e-a1c0-764cf77fb6de None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:15.441336 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7747e6-01a5-4e5e-a1c0-764cf77fb6de None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:15.451415 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4b1cf-4f75-4239-9f9a-777142559a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:15.452636 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4b1cf-4f75-4239-9f9a-777142559a75 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:15.453750 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4b1cf-4f75-4239-9f9a-777142559a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:15.454389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4b1cf-4f75-4239-9f9a-777142559a75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:15.455411 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4b1cf-4f75-4239-9f9a-777142559a75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:15.458055 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4b1cf-4f75-4239-9f9a-777142559a75 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:15.496112 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1195/2360] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:08:15 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:08:15.500023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef391b8-696b-4521-ab33-02044aca99e7 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:15.512474 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef391b8-696b-4521-ab33-02044aca99e7 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:15.589905 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8738e4-c74a-4dce-aaad-191efa81378e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:15.590813 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8738e4-c74a-4dce-aaad-191efa81378e admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:15.591189 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8738e4-c74a-4dce-aaad-191efa81378e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:15.591566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8738e4-c74a-4dce-aaad-191efa81378e admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:15.593015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8738e4-c74a-4dce-aaad-191efa81378e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:15.595303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8738e4-c74a-4dce-aaad-191efa81378e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:15.632581 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1166/2361] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:08:15 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:08:15.634718 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4b1cf-4f75-4239-9f9a-777142559a75 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:15.647969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4b1cf-4f75-4239-9f9a-777142559a75 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:15.652995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786a641-c4ef-4241-8136-6ea456c48fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:15.653915 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786a641-c4ef-4241-8136-6ea456c48fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:15.654852 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786a641-c4ef-4241-8136-6ea456c48fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:15.655327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786a641-c4ef-4241-8136-6ea456c48fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:15.656242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2786a641-c4ef-4241-8136-6ea456c48fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:15.658066 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2786a641-c4ef-4241-8136-6ea456c48fa9 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:15.670230 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1196/2362] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:08:15.673811 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8738e4-c74a-4dce-aaad-191efa81378e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:15.689841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8738e4-c74a-4dce-aaad-191efa81378e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:15.696710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950ae7f-e3dc-469a-90e1-ad74f32cab32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:15.697514 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950ae7f-e3dc-469a-90e1-ad74f32cab32 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:15.697857 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950ae7f-e3dc-469a-90e1-ad74f32cab32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:15.698402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950ae7f-e3dc-469a-90e1-ad74f32cab32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:15.699159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950ae7f-e3dc-469a-90e1-ad74f32cab32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:15.700915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950ae7f-e3dc-469a-90e1-ad74f32cab32 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:15.712848 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1167/2363] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:08:15.717235 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786a641-c4ef-4241-8136-6ea456c48fa9 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:15.729232 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786a641-c4ef-4241-8136-6ea456c48fa9 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:15.734558 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d366fd-a7dd-46ae-8cf1-11c8caae15d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:15.735769 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d366fd-a7dd-46ae-8cf1-11c8caae15d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:15.736112 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d366fd-a7dd-46ae-8cf1-11c8caae15d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:15.736505 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d366fd-a7dd-46ae-8cf1-11c8caae15d8 admin admin] PATH_INFO: `/v3/projects/2bc4f162846047e0a2fa2756c6eecfba/users/01ad27499478423f9ee827741d992191/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:15.761452 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d366fd-a7dd-46ae-8cf1-11c8caae15d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bc4f162846047e0a2fa2756c6eecfba, user_id=01ad27499478423f9ee827741d992191, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:15.764153 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d366fd-a7dd-46ae-8cf1-11c8caae15d8 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:15.767094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:15.785347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1197/2364] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:08:15 2022] PUT /identity/v3/projects/2bc4f162846047e0a2fa2756c6eecfba/users/01ad27499478423f9ee827741d992191/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:15.795495 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4e279-f580-4c7a-9bad-fc73d160000c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:15.795782 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4e279-f580-4c7a-9bad-fc73d160000c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:15.796060 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4e279-f580-4c7a-9bad-fc73d160000c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:15.830618 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-f2c4e279-f580-4c7a-9bad-fc73d160000c None None] MFA Rules not processed for user `01ad27499478423f9ee827741d992191`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:15.832395 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f2c4e279-f580-4c7a-9bad-fc73d160000c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:15.853581 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:15.911969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1168/2365] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:15.923797 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d366fd-a7dd-46ae-8cf1-11c8caae15d8 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:15.925794 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d2d366fd-a7dd-46ae-8cf1-11c8caae15d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:15.985069 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d366fd-a7dd-46ae-8cf1-11c8caae15d8 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:15.988853 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b47867-0341-423a-b296-00cccaa94bbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:15.989683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b47867-0341-423a-b296-00cccaa94bbb service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:15.989962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b47867-0341-423a-b296-00cccaa94bbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:15.990235 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b47867-0341-423a-b296-00cccaa94bbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:15.999104 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b47867-0341-423a-b296-00cccaa94bbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:16.000704 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b47867-0341-423a-b296-00cccaa94bbb service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:16.011318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1198/2366] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:08:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:08:16.575283 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4e279-f580-4c7a-9bad-fc73d160000c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:16.577037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-f2c4e279-f580-4c7a-9bad-fc73d160000c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:16.590985 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4e279-f580-4c7a-9bad-fc73d160000c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:16.594667 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc6ecf-cf6c-40f9-a9c0-da5f14abccff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:16.595620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc6ecf-cf6c-40f9-a9c0-da5f14abccff service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:16.596010 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc6ecf-cf6c-40f9-a9c0-da5f14abccff service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:16.596362 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc6ecf-cf6c-40f9-a9c0-da5f14abccff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:16.598258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4dfc6ecf-cf6c-40f9-a9c0-da5f14abccff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:16.676363 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc6ecf-cf6c-40f9-a9c0-da5f14abccff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:16.678247 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc6ecf-cf6c-40f9-a9c0-da5f14abccff service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:16.694673 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1169/2367] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:08:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:08:17.373998 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b47867-0341-423a-b296-00cccaa94bbb service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:17.383540 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b47867-0341-423a-b296-00cccaa94bbb service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:17.388332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551652a7-fbd2-4b0c-814c-204db5833721 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:17.389022 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551652a7-fbd2-4b0c-814c-204db5833721 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:17.389302 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551652a7-fbd2-4b0c-814c-204db5833721 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:17.389754 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551652a7-fbd2-4b0c-814c-204db5833721 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:17.392147 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-551652a7-fbd2-4b0c-814c-204db5833721 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:17.476860 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551652a7-fbd2-4b0c-814c-204db5833721 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:17.478522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551652a7-fbd2-4b0c-814c-204db5833721 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:17.492927 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1199/2368] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:08:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:08:26.683645 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc6ecf-cf6c-40f9-a9c0-da5f14abccff service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:26.686609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=48 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:26.686609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4dfc6ecf-cf6c-40f9-a9c0-da5f14abccff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:26.758500 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc6ecf-cf6c-40f9-a9c0-da5f14abccff service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:26.763354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585a41d-fab2-4f6b-8c18-547acedf2297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:26.764303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585a41d-fab2-4f6b-8c18-547acedf2297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:26.764640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585a41d-fab2-4f6b-8c18-547acedf2297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:26.765217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585a41d-fab2-4f6b-8c18-547acedf2297 admin admin] PATH_INFO: `/v3/users/c6fc8b2e9c1e49848d9a6536e1e8a30f` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:26.766550 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585a41d-fab2-4f6b-8c18-547acedf2297 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6fc8b2e9c1e49848d9a6536e1e8a30f)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:26.768617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585a41d-fab2-4f6b-8c18-547acedf2297 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:26.824391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5585a41d-fab2-4f6b-8c18-547acedf2297 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6fc8b2e9c1e49848d9a6536e1e8a30f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:26.825138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5585a41d-fab2-4f6b-8c18-547acedf2297 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6fc8b2e9c1e49848d9a6536e1e8a30f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:26.835954 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5585a41d-fab2-4f6b-8c18-547acedf2297 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6fc8b2e9c1e49848d9a6536e1e8a30f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:26.845413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5585a41d-fab2-4f6b-8c18-547acedf2297 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6fc8b2e9c1e49848d9a6536e1e8a30f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:26.846266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5585a41d-fab2-4f6b-8c18-547acedf2297 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6fc8b2e9c1e49848d9a6536e1e8a30f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:26.867134 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1170/2369] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:08:26 2022] DELETE /identity/v3/users/c6fc8b2e9c1e49848d9a6536e1e8a30f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:26.871630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502c9e7-d78c-40a5-a58a-f0e0542bf771 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:26.872068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502c9e7-d78c-40a5-a58a-f0e0542bf771 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:26.872174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502c9e7-d78c-40a5-a58a-f0e0542bf771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:26.889449 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b502c9e7-d78c-40a5-a58a-f0e0542bf771 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:26.890948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b502c9e7-d78c-40a5-a58a-f0e0542bf771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:26.957661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1200/2370] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:26.969165 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585a41d-fab2-4f6b-8c18-547acedf2297 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:26.970745 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5585a41d-fab2-4f6b-8c18-547acedf2297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:27.033722 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585a41d-fab2-4f6b-8c18-547acedf2297 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:27.037755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0be4a9-4a5f-4c33-a045-7ce2b4fbd1db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:27.038409 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0be4a9-4a5f-4c33-a045-7ce2b4fbd1db service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:27.038684 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0be4a9-4a5f-4c33-a045-7ce2b4fbd1db service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:27.038936 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0be4a9-4a5f-4c33-a045-7ce2b4fbd1db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:27.047944 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0be4a9-4a5f-4c33-a045-7ce2b4fbd1db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:27.049492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0be4a9-4a5f-4c33-a045-7ce2b4fbd1db service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:27.060013 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1171/2371] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:08:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:08:27.353229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502c9e7-d78c-40a5-a58a-f0e0542bf771 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:27.355707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b502c9e7-d78c-40a5-a58a-f0e0542bf771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:27.369641 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502c9e7-d78c-40a5-a58a-f0e0542bf771 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:27.376884 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77e9b7-9e1d-4e82-a1f4-6e9ac18bb7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:27.377962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77e9b7-9e1d-4e82-a1f4-6e9ac18bb7f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:27.378394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77e9b7-9e1d-4e82-a1f4-6e9ac18bb7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:27.379007 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77e9b7-9e1d-4e82-a1f4-6e9ac18bb7f4 admin admin] PATH_INFO: `/v3/projects/065373b938b744eeb805f8878e83f716` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:27.380640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77e9b7-9e1d-4e82-a1f4-6e9ac18bb7f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=065373b938b744eeb805f8878e83f716)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:27.382727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77e9b7-9e1d-4e82-a1f4-6e9ac18bb7f4 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:27.429304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:INSERT=1,keystone:SELECT=98,keystone:DELETE=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:27.452311 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-cd77e9b7-9e1d-4e82-a1f4-6e9ac18bb7f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '065373b938b744eeb805f8878e83f716', 'request_id': 'req-cd77e9b7-9e1d-4e82-a1f4-6e9ac18bb7f4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:27.461575 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-cd77e9b7-9e1d-4e82-a1f4-6e9ac18bb7f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '065373b938b744eeb805f8878e83f716', 'request_id': 'req-cd77e9b7-9e1d-4e82-a1f4-6e9ac18bb7f4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:27.467829 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-cd77e9b7-9e1d-4e82-a1f4-6e9ac18bb7f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '065373b938b744eeb805f8878e83f716', 'request_id': 'req-cd77e9b7-9e1d-4e82-a1f4-6e9ac18bb7f4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:27.485142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-cd77e9b7-9e1d-4e82-a1f4-6e9ac18bb7f4 admin admin] The token cache is being invalidate because project 065373b938b744eeb805f8878e83f716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:08:27.485707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-cd77e9b7-9e1d-4e82-a1f4-6e9ac18bb7f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:27.486871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-cd77e9b7-9e1d-4e82-a1f4-6e9ac18bb7f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:27.489020 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1201/2372] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:08:27 2022] DELETE /identity/v3/projects/065373b938b744eeb805f8878e83f716 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:27.493401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0be4a9-4a5f-4c33-a045-7ce2b4fbd1db service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:27.506825 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0be4a9-4a5f-4c33-a045-7ce2b4fbd1db service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:27.533921 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:27.595764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359e883-150c-45ea-b2f6-094c7610b0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:27.596831 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359e883-150c-45ea-b2f6-094c7610b0a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:27.597279 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359e883-150c-45ea-b2f6-094c7610b0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:27.597629 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359e883-150c-45ea-b2f6-094c7610b0a6 admin admin] PATH_INFO: `/v3/users/3a5539386ed24df0ae87b782c376b76c` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:27.599661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359e883-150c-45ea-b2f6-094c7610b0a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a5539386ed24df0ae87b782c376b76c)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:27.601628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359e883-150c-45ea-b2f6-094c7610b0a6 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:27.681464 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6359e883-150c-45ea-b2f6-094c7610b0a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a5539386ed24df0ae87b782c376b76c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:27.682235 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6359e883-150c-45ea-b2f6-094c7610b0a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a5539386ed24df0ae87b782c376b76c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:27.703605 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6359e883-150c-45ea-b2f6-094c7610b0a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a5539386ed24df0ae87b782c376b76c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:27.714124 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6359e883-150c-45ea-b2f6-094c7610b0a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a5539386ed24df0ae87b782c376b76c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:27.718744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6359e883-150c-45ea-b2f6-094c7610b0a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a5539386ed24df0ae87b782c376b76c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:27.749902 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1172/2373] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:08:27 2022] DELETE /identity/v3/users/3a5539386ed24df0ae87b782c376b76c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:28.127133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77e9b7-9e1d-4e82-a1f4-6e9ac18bb7f4 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:28.132078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-cd77e9b7-9e1d-4e82-a1f4-6e9ac18bb7f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:28.159549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77e9b7-9e1d-4e82-a1f4-6e9ac18bb7f4 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:28.240549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d1b3e9-bad7-4ee5-b239-fcc103bd334f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:28.242250 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1b3e9-bad7-4ee5-b239-fcc103bd334f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:28.242712 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1b3e9-bad7-4ee5-b239-fcc103bd334f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:28.243179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1b3e9-bad7-4ee5-b239-fcc103bd334f admin admin] PATH_INFO: `/v3/projects/8d582381453440fda980dd7356d1daff` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:28.245120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d1b3e9-bad7-4ee5-b239-fcc103bd334f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d582381453440fda980dd7356d1daff)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:28.247479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d1b3e9-bad7-4ee5-b239-fcc103bd334f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:28.295144 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a0d1b3e9-bad7-4ee5-b239-fcc103bd334f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d582381453440fda980dd7356d1daff', 'request_id': 'req-a0d1b3e9-bad7-4ee5-b239-fcc103bd334f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:28.301638 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a0d1b3e9-bad7-4ee5-b239-fcc103bd334f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d582381453440fda980dd7356d1daff', 'request_id': 'req-a0d1b3e9-bad7-4ee5-b239-fcc103bd334f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:28.309725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a0d1b3e9-bad7-4ee5-b239-fcc103bd334f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d582381453440fda980dd7356d1daff', 'request_id': 'req-a0d1b3e9-bad7-4ee5-b239-fcc103bd334f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:28.325129 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a0d1b3e9-bad7-4ee5-b239-fcc103bd334f admin admin] The token cache is being invalidate because project 8d582381453440fda980dd7356d1daff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:08:28.325667 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a0d1b3e9-bad7-4ee5-b239-fcc103bd334f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:28.326590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a0d1b3e9-bad7-4ee5-b239-fcc103bd334f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:28.331615 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1202/2374] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:08:28 2022] DELETE /identity/v3/projects/8d582381453440fda980dd7356d1daff => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:28.333219 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359e883-150c-45ea-b2f6-094c7610b0a6 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:28.335135 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6359e883-150c-45ea-b2f6-094c7610b0a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:28.350325 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359e883-150c-45ea-b2f6-094c7610b0a6 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:28.420125 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c3cde-4f1d-47a7-9ed3-6e480c0fcdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:28.421074 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c3cde-4f1d-47a7-9ed3-6e480c0fcdcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:28.421357 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c3cde-4f1d-47a7-9ed3-6e480c0fcdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:28.421635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c3cde-4f1d-47a7-9ed3-6e480c0fcdcf admin admin] PATH_INFO: `/v3/users/b9ccf584cbc1452ea1bc1e7b7dd33afa` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:28.422846 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c3cde-4f1d-47a7-9ed3-6e480c0fcdcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9ccf584cbc1452ea1bc1e7b7dd33afa)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:28.424999 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c3cde-4f1d-47a7-9ed3-6e480c0fcdcf admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:28.498282 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:28.501001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f95c3cde-4f1d-47a7-9ed3-6e480c0fcdcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9ccf584cbc1452ea1bc1e7b7dd33afa'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:28.501763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f95c3cde-4f1d-47a7-9ed3-6e480c0fcdcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9ccf584cbc1452ea1bc1e7b7dd33afa'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:28.511887 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f95c3cde-4f1d-47a7-9ed3-6e480c0fcdcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9ccf584cbc1452ea1bc1e7b7dd33afa'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:28.520376 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f95c3cde-4f1d-47a7-9ed3-6e480c0fcdcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9ccf584cbc1452ea1bc1e7b7dd33afa'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:28.521263 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-f95c3cde-4f1d-47a7-9ed3-6e480c0fcdcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9ccf584cbc1452ea1bc1e7b7dd33afa'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:28.542255 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1173/2375] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:08:28 2022] DELETE /identity/v3/users/b9ccf584cbc1452ea1bc1e7b7dd33afa => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:29.206046 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d1b3e9-bad7-4ee5-b239-fcc103bd334f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:29.207841 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a0d1b3e9-bad7-4ee5-b239-fcc103bd334f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:29.221042 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d1b3e9-bad7-4ee5-b239-fcc103bd334f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:29.270322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:29.287789 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2940f11b-810b-4566-beda-fd0191ea1980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:29.288707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940f11b-810b-4566-beda-fd0191ea1980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:29.288993 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940f11b-810b-4566-beda-fd0191ea1980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:29.289270 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2940f11b-810b-4566-beda-fd0191ea1980 admin admin] PATH_INFO: `/v3/projects/9e5f6486d9b041d2b8d0ef95c48fdba0` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:29.290494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2940f11b-810b-4566-beda-fd0191ea1980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e5f6486d9b041d2b8d0ef95c48fdba0)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:29.292439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2940f11b-810b-4566-beda-fd0191ea1980 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:29.346581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2940f11b-810b-4566-beda-fd0191ea1980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e5f6486d9b041d2b8d0ef95c48fdba0', 'request_id': 'req-2940f11b-810b-4566-beda-fd0191ea1980'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:29.352338 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2940f11b-810b-4566-beda-fd0191ea1980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e5f6486d9b041d2b8d0ef95c48fdba0', 'request_id': 'req-2940f11b-810b-4566-beda-fd0191ea1980'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:29.358799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2940f11b-810b-4566-beda-fd0191ea1980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e5f6486d9b041d2b8d0ef95c48fdba0', 'request_id': 'req-2940f11b-810b-4566-beda-fd0191ea1980'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:29.372927 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2940f11b-810b-4566-beda-fd0191ea1980 admin admin] The token cache is being invalidate because project 9e5f6486d9b041d2b8d0ef95c48fdba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:08:29.373215 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2940f11b-810b-4566-beda-fd0191ea1980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:29.373834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2940f11b-810b-4566-beda-fd0191ea1980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:29.378753 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1203/2376] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:08:29 2022] DELETE /identity/v3/projects/9e5f6486d9b041d2b8d0ef95c48fdba0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:29.384891 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8077a-56ff-49b2-aba5-74df0dff4aa3 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:29.385432 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8077a-56ff-49b2-aba5-74df0dff4aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:29.385555 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8077a-56ff-49b2-aba5-74df0dff4aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:29.406184 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-50f8077a-56ff-49b2-aba5-74df0dff4aa3 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:29.407411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-50f8077a-56ff-49b2-aba5-74df0dff4aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:29.472378 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1174/2377] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:29.480145 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c2c7f-60fb-4741-b56b-8ea94116eee2 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:29.480532 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c2c7f-60fb-4741-b56b-8ea94116eee2 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:29.480903 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c2c7f-60fb-4741-b56b-8ea94116eee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:29.499823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-307c2c7f-60fb-4741-b56b-8ea94116eee2 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:29.501068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-307c2c7f-60fb-4741-b56b-8ea94116eee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:29.509041 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1204/2378] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:29.513401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8077a-56ff-49b2-aba5-74df0dff4aa3 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:29.524440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8077a-56ff-49b2-aba5-74df0dff4aa3 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:29.529282 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0ea9e-538a-4dd5-bc77-676351a984ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:29.530044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0ea9e-538a-4dd5-bc77-676351a984ca admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:29.530394 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0ea9e-538a-4dd5-bc77-676351a984ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:29.530730 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0ea9e-538a-4dd5-bc77-676351a984ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:29.531571 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0ea9e-538a-4dd5-bc77-676351a984ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:08:29.531933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0ea9e-538a-4dd5-bc77-676351a984ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:29.533646 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0ea9e-538a-4dd5-bc77-676351a984ca admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:29.546390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1175/2379] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:08:29.550974 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c2c7f-60fb-4741-b56b-8ea94116eee2 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:29.560673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c2c7f-60fb-4741-b56b-8ea94116eee2 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:29.565474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f960a6-5662-4849-883b-7f648cab8c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:29.566132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f960a6-5662-4849-883b-7f648cab8c78 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:29.566409 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f960a6-5662-4849-883b-7f648cab8c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:29.566680 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f960a6-5662-4849-883b-7f648cab8c78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:29.567705 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f960a6-5662-4849-883b-7f648cab8c78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:29.569621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f960a6-5662-4849-883b-7f648cab8c78 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:29.604966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1205/2380] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:08:29 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:08:29.609034 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0ea9e-538a-4dd5-bc77-676351a984ca admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:29.625133 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0ea9e-538a-4dd5-bc77-676351a984ca admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:29.685842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd4f2a-5259-4feb-bf9b-44dbc2f5c000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:29.686673 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd4f2a-5259-4feb-bf9b-44dbc2f5c000 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:29.687019 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd4f2a-5259-4feb-bf9b-44dbc2f5c000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:29.687328 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd4f2a-5259-4feb-bf9b-44dbc2f5c000 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:29.688179 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd4f2a-5259-4feb-bf9b-44dbc2f5c000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:29.690160 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd4f2a-5259-4feb-bf9b-44dbc2f5c000 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:29.711989 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1176/2381] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:08:29 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:08:29.717760 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f960a6-5662-4849-883b-7f648cab8c78 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:29.729196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f960a6-5662-4849-883b-7f648cab8c78 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:29.733944 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f01bf5-e982-4adb-9684-10b161cc2de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:29.734613 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f01bf5-e982-4adb-9684-10b161cc2de2 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:29.734893 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f01bf5-e982-4adb-9684-10b161cc2de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:29.735166 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f01bf5-e982-4adb-9684-10b161cc2de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:29.735894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f01bf5-e982-4adb-9684-10b161cc2de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:29.737545 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f01bf5-e982-4adb-9684-10b161cc2de2 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:29.748756 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1206/2382] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:08:29.753065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd4f2a-5259-4feb-bf9b-44dbc2f5c000 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:29.763698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd4f2a-5259-4feb-bf9b-44dbc2f5c000 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:29.769226 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be94c2f-a5c0-462e-ac54-b39b210fa17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:29.769893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be94c2f-a5c0-462e-ac54-b39b210fa17d admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:29.770506 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be94c2f-a5c0-462e-ac54-b39b210fa17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:29.770506 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be94c2f-a5c0-462e-ac54-b39b210fa17d admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:29.771272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be94c2f-a5c0-462e-ac54-b39b210fa17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:29.772892 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be94c2f-a5c0-462e-ac54-b39b210fa17d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:29.784352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1177/2383] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:08:29.789404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f01bf5-e982-4adb-9684-10b161cc2de2 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:29.802904 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f01bf5-e982-4adb-9684-10b161cc2de2 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:29.808232 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2d7ac-7122-4af5-86ea-967f02ac3d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:29.809365 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2d7ac-7122-4af5-86ea-967f02ac3d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:29.809657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2d7ac-7122-4af5-86ea-967f02ac3d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:29.809934 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2d7ac-7122-4af5-86ea-967f02ac3d16 admin admin] PATH_INFO: `/v3/projects/be35accfa19641058c6452311878f2d4/users/bbf2525354424683bfa550eadc0a748c/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:29.826147 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2d7ac-7122-4af5-86ea-967f02ac3d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be35accfa19641058c6452311878f2d4, user_id=bbf2525354424683bfa550eadc0a748c, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:29.828860 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2d7ac-7122-4af5-86ea-967f02ac3d16 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:29.870174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1207/2384] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:08:29 2022] PUT /identity/v3/projects/be35accfa19641058c6452311878f2d4/users/bbf2525354424683bfa550eadc0a748c/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:29.873503 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19725b3-ea1c-4dc3-90d0-5c93eeabf65b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:29.873795 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19725b3-ea1c-4dc3-90d0-5c93eeabf65b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:29.874079 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19725b3-ea1c-4dc3-90d0-5c93eeabf65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:29.893546 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c19725b3-ea1c-4dc3-90d0-5c93eeabf65b None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:29.895026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c19725b3-ea1c-4dc3-90d0-5c93eeabf65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:29.923871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:29.966135 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1178/2385] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:29.975901 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2d7ac-7122-4af5-86ea-967f02ac3d16 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:29.977852 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f3e2d7ac-7122-4af5-86ea-967f02ac3d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:30.052190 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2d7ac-7122-4af5-86ea-967f02ac3d16 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:30.056190 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60231057-443d-4221-9e95-27adeb90b020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:30.056874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60231057-443d-4221-9e95-27adeb90b020 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:30.057156 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60231057-443d-4221-9e95-27adeb90b020 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:30.057402 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60231057-443d-4221-9e95-27adeb90b020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:30.068553 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60231057-443d-4221-9e95-27adeb90b020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:30.070035 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60231057-443d-4221-9e95-27adeb90b020 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:30.080208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1208/2386] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:08:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:08:30.949427 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77529ea1-1486-4785-9c57-a00022b7f6b4 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:30.949922 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77529ea1-1486-4785-9c57-a00022b7f6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:30.950071 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77529ea1-1486-4785-9c57-a00022b7f6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:30.979112 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-77529ea1-1486-4785-9c57-a00022b7f6b4 None None] MFA Rules not processed for user `bbf2525354424683bfa550eadc0a748c`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:30.980329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-77529ea1-1486-4785-9c57-a00022b7f6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:31.065775 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1179/2387] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:31.077734 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60231057-443d-4221-9e95-27adeb90b020 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:31.079294 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-60231057-443d-4221-9e95-27adeb90b020 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:31.093774 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:31.152052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60231057-443d-4221-9e95-27adeb90b020 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:31.156059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77a716-ad23-44a0-9627-6939114e7927 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:31.156754 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77a716-ad23-44a0-9627-6939114e7927 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:31.157059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77a716-ad23-44a0-9627-6939114e7927 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:31.157349 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77a716-ad23-44a0-9627-6939114e7927 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:31.166578 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77a716-ad23-44a0-9627-6939114e7927 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:31.167994 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77a716-ad23-44a0-9627-6939114e7927 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:31.177586 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1209/2388] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:08:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:08:32.576660 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17096b-e4e5-4c4c-9e18-a969e75aea79 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:32.577210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17096b-e4e5-4c4c-9e18-a969e75aea79 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:32.577210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17096b-e4e5-4c4c-9e18-a969e75aea79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:32.595814 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-6d17096b-e4e5-4c4c-9e18-a969e75aea79 None None] MFA Rules not processed for user `bbf2525354424683bfa550eadc0a748c`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:32.597108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6d17096b-e4e5-4c4c-9e18-a969e75aea79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:32.605138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1180/2389] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:37.173765 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77a716-ad23-44a0-9627-6939114e7927 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:37.175224 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5f77a716-ad23-44a0-9627-6939114e7927 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:37.196897 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77a716-ad23-44a0-9627-6939114e7927 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:37.202850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1daa84e-1174-4acf-9953-32c898dd54e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:37.203740 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1daa84e-1174-4acf-9953-32c898dd54e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:37.204017 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1daa84e-1174-4acf-9953-32c898dd54e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:37.204298 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1daa84e-1174-4acf-9953-32c898dd54e3 admin admin] PATH_INFO: `/v3/users/01ad27499478423f9ee827741d992191` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:37.205769 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1daa84e-1174-4acf-9953-32c898dd54e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01ad27499478423f9ee827741d992191)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:37.207837 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1daa84e-1174-4acf-9953-32c898dd54e3 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:37.281795 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f1daa84e-1174-4acf-9953-32c898dd54e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01ad27499478423f9ee827741d992191'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:37.298192 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f1daa84e-1174-4acf-9953-32c898dd54e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01ad27499478423f9ee827741d992191'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:37.298964 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f1daa84e-1174-4acf-9953-32c898dd54e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01ad27499478423f9ee827741d992191'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:37.310131 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f1daa84e-1174-4acf-9953-32c898dd54e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01ad27499478423f9ee827741d992191'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:37.310806 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f1daa84e-1174-4acf-9953-32c898dd54e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01ad27499478423f9ee827741d992191'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:37.333249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1210/2390] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:08:37 2022] DELETE /identity/v3/users/01ad27499478423f9ee827741d992191 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:37.338306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1e6b2-9dd5-447b-b6a0-ec7cbee398e9 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:37.338792 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1e6b2-9dd5-447b-b6a0-ec7cbee398e9 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:37.338886 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1e6b2-9dd5-447b-b6a0-ec7cbee398e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:37.358750 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-15a1e6b2-9dd5-447b-b6a0-ec7cbee398e9 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:08:37.360369 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-15a1e6b2-9dd5-447b-b6a0-ec7cbee398e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:37.433152 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1181/2391] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:08:37.443122 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1daa84e-1174-4acf-9953-32c898dd54e3 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:37.444970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f1daa84e-1174-4acf-9953-32c898dd54e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:37.507959 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1daa84e-1174-4acf-9953-32c898dd54e3 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:37.511835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c63bc2-8ea3-4937-b43a-10ce920bc4b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:37.512597 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c63bc2-8ea3-4937-b43a-10ce920bc4b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:37.512954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c63bc2-8ea3-4937-b43a-10ce920bc4b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:37.513298 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c63bc2-8ea3-4937-b43a-10ce920bc4b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:37.524773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c63bc2-8ea3-4937-b43a-10ce920bc4b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:37.526473 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c63bc2-8ea3-4937-b43a-10ce920bc4b4 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:37.538081 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1211/2392] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:08:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:08:37.865986 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1e6b2-9dd5-447b-b6a0-ec7cbee398e9 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:37.867821 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-15a1e6b2-9dd5-447b-b6a0-ec7cbee398e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:37.881137 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1e6b2-9dd5-447b-b6a0-ec7cbee398e9 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:37.886898 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92edc4-8ae3-4da5-8169-b27da6ce38f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:37.888123 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92edc4-8ae3-4da5-8169-b27da6ce38f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:37.888505 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92edc4-8ae3-4da5-8169-b27da6ce38f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:37.888847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92edc4-8ae3-4da5-8169-b27da6ce38f7 admin admin] PATH_INFO: `/v3/projects/2bc4f162846047e0a2fa2756c6eecfba` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:37.891395 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92edc4-8ae3-4da5-8169-b27da6ce38f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bc4f162846047e0a2fa2756c6eecfba)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:37.893542 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92edc4-8ae3-4da5-8169-b27da6ce38f7 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:08:37.955226 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:37.958072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6a92edc4-8ae3-4da5-8169-b27da6ce38f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bc4f162846047e0a2fa2756c6eecfba', 'request_id': 'req-6a92edc4-8ae3-4da5-8169-b27da6ce38f7'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:37.969789 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6a92edc4-8ae3-4da5-8169-b27da6ce38f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bc4f162846047e0a2fa2756c6eecfba', 'request_id': 'req-6a92edc4-8ae3-4da5-8169-b27da6ce38f7'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:37.978868 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6a92edc4-8ae3-4da5-8169-b27da6ce38f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bc4f162846047e0a2fa2756c6eecfba', 'request_id': 'req-6a92edc4-8ae3-4da5-8169-b27da6ce38f7'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:38.002515 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6a92edc4-8ae3-4da5-8169-b27da6ce38f7 admin admin] The token cache is being invalidate because project 2bc4f162846047e0a2fa2756c6eecfba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:08:38.003796 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6a92edc4-8ae3-4da5-8169-b27da6ce38f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:38.006609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-6a92edc4-8ae3-4da5-8169-b27da6ce38f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:08:38.028210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1182/2393] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:08:37 2022] DELETE /identity/v3/projects/2bc4f162846047e0a2fa2756c6eecfba => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:08:47.533514 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=75,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:47.997828 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:08:59.764649 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c63bc2-8ea3-4937-b43a-10ce920bc4b4 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:08:59.766472 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-49c63bc2-8ea3-4937-b43a-10ce920bc4b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:59.842553 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c63bc2-8ea3-4937-b43a-10ce920bc4b4 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:08:59.905506 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6556f5-d553-4fdb-b845-4138c5bf3668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:08:59.906334 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6556f5-d553-4fdb-b845-4138c5bf3668 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:08:59.906498 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6556f5-d553-4fdb-b845-4138c5bf3668 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:08:59.907368 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6556f5-d553-4fdb-b845-4138c5bf3668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:08:59.909442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-cf6556f5-d553-4fdb-b845-4138c5bf3668 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:08:59.993519 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6556f5-d553-4fdb-b845-4138c5bf3668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:08:59.994962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6556f5-d553-4fdb-b845-4138c5bf3668 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:09:00.005034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1212/2394] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:08:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:09:00.625319 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f418db-b7c6-492d-b583-595e17467024 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:00.625895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f418db-b7c6-492d-b583-595e17467024 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:00.625895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f418db-b7c6-492d-b583-595e17467024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:00.645283 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e1f418db-b7c6-492d-b583-595e17467024 None None] MFA Rules not processed for user `bbf2525354424683bfa550eadc0a748c`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:09:00.646606 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e1f418db-b7c6-492d-b583-595e17467024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:09:00.714961 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1183/2395] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:09:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:09:10.000307 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=81 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:09:10.707993 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=25 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:09:17.722177 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90614bb9-0930-4fe7-87f9-358de032e512 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:17.722690 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90614bb9-0930-4fe7-87f9-358de032e512 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:17.722970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90614bb9-0930-4fe7-87f9-358de032e512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:17.741689 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-90614bb9-0930-4fe7-87f9-358de032e512 None None] MFA Rules not processed for user `bf0332a9562946169ff8a45bf5e6f6f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:09:17.742917 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-90614bb9-0930-4fe7-87f9-358de032e512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:09:17.810038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1213/2396] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:09:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:09:17.901666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f418db-b7c6-492d-b583-595e17467024 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:09:17.903206 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e1f418db-b7c6-492d-b583-595e17467024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:09:18.013575 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f418db-b7c6-492d-b583-595e17467024 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:09:18.019333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf358cc-7232-4cca-b955-cc35f121aaf5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474c468a610140f8a5b18d5a1ed1d18e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:09:18.020086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf358cc-7232-4cca-b955-cc35f121aaf5 service placement] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:18.020456 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf358cc-7232-4cca-b955-cc35f121aaf5 service placement] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:18.020769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf358cc-7232-4cca-b955-cc35f121aaf5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:18.022601 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-dbf358cc-7232-4cca-b955-cc35f121aaf5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:09:18.034948 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf358cc-7232-4cca-b955-cc35f121aaf5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:09:18.036895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf358cc-7232-4cca-b955-cc35f121aaf5 service placement] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:09:18.047137 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1184/2397] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:09:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:09:26.780846 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f566439-1b7a-4d97-bb0c-bf578205122a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:26.781336 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f566439-1b7a-4d97-bb0c-bf578205122a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:26.781429 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f566439-1b7a-4d97-bb0c-bf578205122a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:26.810856 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7f566439-1b7a-4d97-bb0c-bf578205122a None None] MFA Rules not processed for user `28b3e73439e64050b406a9f3e9b0160a`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:09:26.812085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7f566439-1b7a-4d97-bb0c-bf578205122a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:09:26.880073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1214/2398] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Jun 25 16:09:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:09:26.922360 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf358cc-7232-4cca-b955-cc35f121aaf5 service placement] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:09:26.924597 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-dbf358cc-7232-4cca-b955-cc35f121aaf5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:09:26.998762 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf358cc-7232-4cca-b955-cc35f121aaf5 service placement] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:09:27.004852 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8daffcd-172a-4f9d-8cdd-3b7b5f5b1710 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19788c4239ca44f9a31a7d32086c8947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:09:27.005494 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8daffcd-172a-4f9d-8cdd-3b7b5f5b1710 service swift] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:27.005771 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8daffcd-172a-4f9d-8cdd-3b7b5f5b1710 service swift] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:27.006042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8daffcd-172a-4f9d-8cdd-3b7b5f5b1710 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:27.014509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8daffcd-172a-4f9d-8cdd-3b7b5f5b1710 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:09:27.016431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8daffcd-172a-4f9d-8cdd-3b7b5f5b1710 service swift] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:09:27.025307 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1185/2399] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Jun 25 16:09:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 25 16:09:36.874258 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=57 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:09:37.021668 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=73 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:09:43.367974 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df77c8-7bc0-4d05-9c9f-57b5d1506032 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:43.368693 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df77c8-7bc0-4d05-9c9f-57b5d1506032 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:43.368693 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df77c8-7bc0-4d05-9c9f-57b5d1506032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:43.387877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-16df77c8-7bc0-4d05-9c9f-57b5d1506032 None None] MFA Rules not processed for user `19140330ae664e45b85f47f321496191`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:09:43.389329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-16df77c8-7bc0-4d05-9c9f-57b5d1506032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:09:43.460205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1215/2400] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:09:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:09:47.510542 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47e8ca-8bcd-4d9d-a2fe-c6506d7ae0f1 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:47.511051 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47e8ca-8bcd-4d9d-a2fe-c6506d7ae0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:47.511179 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47e8ca-8bcd-4d9d-a2fe-c6506d7ae0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:47.576926 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5c47e8ca-8bcd-4d9d-a2fe-c6506d7ae0f1 None None] MFA Rules not processed for user `28b3e73439e64050b406a9f3e9b0160a`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:09:47.581367 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5c47e8ca-8bcd-4d9d-a2fe-c6506d7ae0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:09:47.586848 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abba89-4e12-4b7e-a9a5-318932ca4cae None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:47.587360 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abba89-4e12-4b7e-a9a5-318932ca4cae None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:47.587606 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abba89-4e12-4b7e-a9a5-318932ca4cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:47.604948 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1186/2401] 10.222.0.13 () {58 vars in 967 bytes} [Sat Jun 25 16:09:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:09:47.607910 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-68abba89-4e12-4b7e-a9a5-318932ca4cae None None] MFA Rules not processed for user `28b3e73439e64050b406a9f3e9b0160a`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:09:47.609310 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-68abba89-4e12-4b7e-a9a5-318932ca4cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:09:47.617954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1216/2402] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Jun 25 16:09:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:09:47.619986 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47e8ca-8bcd-4d9d-a2fe-c6506d7ae0f1 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:09:47.632984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68abba89-4e12-4b7e-a9a5-318932ca4cae None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:09:47.652890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47e8ca-8bcd-4d9d-a2fe-c6506d7ae0f1 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:09:47.660313 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c9787-a054-40d0-9a05-085505f64f06 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19788c4239ca44f9a31a7d32086c8947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:09:47.661858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c9787-a054-40d0-9a05-085505f64f06 service swift] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:47.662338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c9787-a054-40d0-9a05-085505f64f06 service swift] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:47.662806 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c9787-a054-40d0-9a05-085505f64f06 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:47.662961 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68abba89-4e12-4b7e-a9a5-318932ca4cae None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:09:47.668736 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd201057-94d0-43e0-9365-6eb12921935a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19788c4239ca44f9a31a7d32086c8947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:09:47.669695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd201057-94d0-43e0-9365-6eb12921935a service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:47.670498 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd201057-94d0-43e0-9365-6eb12921935a service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:47.670927 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd201057-94d0-43e0-9365-6eb12921935a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:47.675726 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c9787-a054-40d0-9a05-085505f64f06 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:09:47.678126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c9787-a054-40d0-9a05-085505f64f06 service swift] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:09:47.682962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd201057-94d0-43e0-9365-6eb12921935a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:09:47.685131 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd201057-94d0-43e0-9365-6eb12921935a service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:09:47.699181 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1217/2403] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Jun 25 16:09:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 25 16:09:47.708233 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1187/2404] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Jun 25 16:09:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 25 16:09:52.314530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd201057-94d0-43e0-9365-6eb12921935a service swift] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:09:52.317445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-bd201057-94d0-43e0-9365-6eb12921935a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:09:52.464170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd201057-94d0-43e0-9365-6eb12921935a service swift] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:09:52.469823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb25af-7188-42ed-91c7-ff954dad163d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:09:52.470954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb25af-7188-42ed-91c7-ff954dad163d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:52.471265 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb25af-7188-42ed-91c7-ff954dad163d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:52.471537 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb25af-7188-42ed-91c7-ff954dad163d admin admin] PATH_INFO: `/v3/users/bbf2525354424683bfa550eadc0a748c` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:52.472879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fb25af-7188-42ed-91c7-ff954dad163d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbf2525354424683bfa550eadc0a748c)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:09:52.477192 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fb25af-7188-42ed-91c7-ff954dad163d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:09:52.548857 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a4fb25af-7188-42ed-91c7-ff954dad163d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbf2525354424683bfa550eadc0a748c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:09:52.553159 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:09:52.577751 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a4fb25af-7188-42ed-91c7-ff954dad163d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbf2525354424683bfa550eadc0a748c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:09:52.585258 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a4fb25af-7188-42ed-91c7-ff954dad163d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbf2525354424683bfa550eadc0a748c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:09:52.600104 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a4fb25af-7188-42ed-91c7-ff954dad163d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbf2525354424683bfa550eadc0a748c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:09:52.600793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-a4fb25af-7188-42ed-91c7-ff954dad163d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbf2525354424683bfa550eadc0a748c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:09:52.628873 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1218/2405] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:09:52 2022] DELETE /identity/v3/users/bbf2525354424683bfa550eadc0a748c => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:09:52.824808 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c9787-a054-40d0-9a05-085505f64f06 service swift] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:09:52.839041 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c9787-a054-40d0-9a05-085505f64f06 service swift] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:09:52.924497 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e104ce-85df-44b0-8534-2b066dc178b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:09:52.925596 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e104ce-85df-44b0-8534-2b066dc178b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:52.926054 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e104ce-85df-44b0-8534-2b066dc178b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:52.926508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e104ce-85df-44b0-8534-2b066dc178b6 admin admin] PATH_INFO: `/v3/projects/be35accfa19641058c6452311878f2d4` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:52.928481 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e104ce-85df-44b0-8534-2b066dc178b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be35accfa19641058c6452311878f2d4)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:09:52.931592 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e104ce-85df-44b0-8534-2b066dc178b6 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:09:53.007469 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-84e104ce-85df-44b0-8534-2b066dc178b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be35accfa19641058c6452311878f2d4', 'request_id': 'req-84e104ce-85df-44b0-8534-2b066dc178b6'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:09:53.013562 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-84e104ce-85df-44b0-8534-2b066dc178b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be35accfa19641058c6452311878f2d4', 'request_id': 'req-84e104ce-85df-44b0-8534-2b066dc178b6'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:09:53.022652 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-84e104ce-85df-44b0-8534-2b066dc178b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be35accfa19641058c6452311878f2d4', 'request_id': 'req-84e104ce-85df-44b0-8534-2b066dc178b6'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:09:53.044950 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-84e104ce-85df-44b0-8534-2b066dc178b6 admin admin] The token cache is being invalidate because project be35accfa19641058c6452311878f2d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:09:53.045228 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-84e104ce-85df-44b0-8534-2b066dc178b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:09:53.045897 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-84e104ce-85df-44b0-8534-2b066dc178b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:09:53.047793 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1188/2406] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:09:52 2022] DELETE /identity/v3/projects/be35accfa19641058c6452311878f2d4 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:09:53.052803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33afc7-91b0-4d9c-a6e2-9d57d847feb2 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:53.053108 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33afc7-91b0-4d9c-a6e2-9d57d847feb2 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:53.053350 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33afc7-91b0-4d9c-a6e2-9d57d847feb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:53.125286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ad33afc7-91b0-4d9c-a6e2-9d57d847feb2 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:09:53.126499 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ad33afc7-91b0-4d9c-a6e2-9d57d847feb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:09:53.245525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1219/2407] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:09:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:09:53.253407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb6abf-b188-42dc-b8e2-e0d48be51603 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:53.253739 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb6abf-b188-42dc-b8e2-e0d48be51603 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:53.254074 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb6abf-b188-42dc-b8e2-e0d48be51603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:53.282289 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-19fb6abf-b188-42dc-b8e2-e0d48be51603 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:09:53.284109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-19fb6abf-b188-42dc-b8e2-e0d48be51603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:09:53.313006 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1189/2408] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:09:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:09:53.316996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad33afc7-91b0-4d9c-a6e2-9d57d847feb2 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:09:53.328154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad33afc7-91b0-4d9c-a6e2-9d57d847feb2 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:09:53.334636 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9b7e5-8c87-4b9c-a210-ff1622437f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:09:53.335312 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9b7e5-8c87-4b9c-a210-ff1622437f6b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:53.335591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9b7e5-8c87-4b9c-a210-ff1622437f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:53.336466 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9b7e5-8c87-4b9c-a210-ff1622437f6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:53.336623 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9b7e5-8c87-4b9c-a210-ff1622437f6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:09:53.336929 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9b7e5-8c87-4b9c-a210-ff1622437f6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:09:53.338537 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9b7e5-8c87-4b9c-a210-ff1622437f6b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:09:53.355277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1220/2409] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:09:53.356540 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb6abf-b188-42dc-b8e2-e0d48be51603 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:09:53.371498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb6abf-b188-42dc-b8e2-e0d48be51603 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:09:53.376462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24faa349-4fc7-442a-9333-deff0f85b143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:09:53.377442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24faa349-4fc7-442a-9333-deff0f85b143 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:53.377801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24faa349-4fc7-442a-9333-deff0f85b143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:53.378145 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24faa349-4fc7-442a-9333-deff0f85b143 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:53.378914 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24faa349-4fc7-442a-9333-deff0f85b143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:09:53.381022 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24faa349-4fc7-442a-9333-deff0f85b143 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:09:53.420985 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:09:53.432576 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1190/2410] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:09:53 2022] POST /identity/v3/projects => generated 363 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:09:53.433807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9b7e5-8c87-4b9c-a210-ff1622437f6b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:09:53.446802 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9b7e5-8c87-4b9c-a210-ff1622437f6b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:09:53.553824 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20056c4b-ecf0-46fc-9a24-31ba1fe7c058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:09:53.554530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20056c4b-ecf0-46fc-9a24-31ba1fe7c058 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:53.554815 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20056c4b-ecf0-46fc-9a24-31ba1fe7c058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:53.555273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20056c4b-ecf0-46fc-9a24-31ba1fe7c058 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:53.555993 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20056c4b-ecf0-46fc-9a24-31ba1fe7c058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:09:53.558347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20056c4b-ecf0-46fc-9a24-31ba1fe7c058 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:09:53.587982 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1221/2411] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:09:53 2022] POST /identity/v3/users => generated 282 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:09:53.592747 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24faa349-4fc7-442a-9333-deff0f85b143 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:09:53.618588 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24faa349-4fc7-442a-9333-deff0f85b143 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:09:53.626610 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55b659-aedd-4e95-ac6d-6f3d7764f4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:09:53.632301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55b659-aedd-4e95-ac6d-6f3d7764f4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:53.633141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55b659-aedd-4e95-ac6d-6f3d7764f4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:53.633651 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55b659-aedd-4e95-ac6d-6f3d7764f4b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:53.634629 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c55b659-aedd-4e95-ac6d-6f3d7764f4b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:09:53.636671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c55b659-aedd-4e95-ac6d-6f3d7764f4b7 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:09:53.650225 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1191/2412] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:09:53.655409 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20056c4b-ecf0-46fc-9a24-31ba1fe7c058 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:09:53.671195 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20056c4b-ecf0-46fc-9a24-31ba1fe7c058 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:09:53.676344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9ef77-63bd-4aac-8efe-d27f46d9e261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:09:53.677044 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9ef77-63bd-4aac-8efe-d27f46d9e261 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:53.677347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9ef77-63bd-4aac-8efe-d27f46d9e261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:53.677764 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9ef77-63bd-4aac-8efe-d27f46d9e261 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:53.678524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9ef77-63bd-4aac-8efe-d27f46d9e261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:09:53.680219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9ef77-63bd-4aac-8efe-d27f46d9e261 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:09:53.692975 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1222/2413] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:09:53.697554 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55b659-aedd-4e95-ac6d-6f3d7764f4b7 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:09:53.712827 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55b659-aedd-4e95-ac6d-6f3d7764f4b7 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:09:53.734297 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b06934-4ed1-4a2b-8fec-4f4fd47a9d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:09:53.735270 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b06934-4ed1-4a2b-8fec-4f4fd47a9d62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:53.735635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b06934-4ed1-4a2b-8fec-4f4fd47a9d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:53.735920 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b06934-4ed1-4a2b-8fec-4f4fd47a9d62 admin admin] PATH_INFO: `/v3/projects/c18fde29935c4e3e92999ffa80fce238/users/b7214c24b8fb4bf9b7804b15114b938b/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:53.755368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b06934-4ed1-4a2b-8fec-4f4fd47a9d62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c18fde29935c4e3e92999ffa80fce238, user_id=b7214c24b8fb4bf9b7804b15114b938b, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:09:53.758099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b06934-4ed1-4a2b-8fec-4f4fd47a9d62 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:09:53.775379 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1192/2414] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:09:53 2022] PUT /identity/v3/projects/c18fde29935c4e3e92999ffa80fce238/users/b7214c24b8fb4bf9b7804b15114b938b/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:09:53.786037 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6d9b0-f776-4380-8cd4-7cb190c37e46 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:53.786326 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6d9b0-f776-4380-8cd4-7cb190c37e46 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:53.786609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6d9b0-f776-4380-8cd4-7cb190c37e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:53.793705 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:09:53.826292 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-77d6d9b0-f776-4380-8cd4-7cb190c37e46 None None] MFA Rules not processed for user `b7214c24b8fb4bf9b7804b15114b938b`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:09:53.827612 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-77d6d9b0-f776-4380-8cd4-7cb190c37e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:09:53.906213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1223/2415] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:09:53 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:09:53.919813 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b06934-4ed1-4a2b-8fec-4f4fd47a9d62 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:09:53.922688 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e1b06934-4ed1-4a2b-8fec-4f4fd47a9d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:09:54.037427 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b06934-4ed1-4a2b-8fec-4f4fd47a9d62 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:09:54.041890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00beeed-10ca-42e4-9f3c-4b934e81d06e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:09:54.042755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00beeed-10ca-42e4-9f3c-4b934e81d06e service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:09:54.043150 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00beeed-10ca-42e4-9f3c-4b934e81d06e service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:09:54.044520 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00beeed-10ca-42e4-9f3c-4b934e81d06e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:09:54.056794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00beeed-10ca-42e4-9f3c-4b934e81d06e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:09:54.058880 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00beeed-10ca-42e4-9f3c-4b934e81d06e service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:09:54.073985 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1193/2416] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:09:53 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:10:03.902930 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=31 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:10:04.065971 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=50,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:10:06.486608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f54b3-14e9-4520-81a6-daccce78cdac None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:06.487153 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f54b3-14e9-4520-81a6-daccce78cdac None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:06.487316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f54b3-14e9-4520-81a6-daccce78cdac None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:06.489138 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1224/2417] 10.222.0.13 () {54 vars in 921 bytes} [Sat Jun 25 16:10:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:10:06.496282 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea7d3b-5dc4-4b2c-87dc-ce7bd26a31f5 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:06.496866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea7d3b-5dc4-4b2c-87dc-ce7bd26a31f5 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:06.496971 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea7d3b-5dc4-4b2c-87dc-ce7bd26a31f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:06.548144 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d0ea7d3b-5dc4-4b2c-87dc-ce7bd26a31f5 None None] MFA Rules not processed for user `3622c5ccb7534e7c8e8342b4f6643bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:10:06.549427 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d0ea7d3b-5dc4-4b2c-87dc-ce7bd26a31f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:10:06.631707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1194/2418] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Jun 25 16:10:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:10:06.643984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f54b3-14e9-4520-81a6-daccce78cdac None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:10:06.645942 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-563f54b3-14e9-4520-81a6-daccce78cdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:10:06.737939 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f54b3-14e9-4520-81a6-daccce78cdac None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:10:06.742301 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa60e14-b7f3-4e14-a2d6-190a57e46231 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cba086728824ec68fb0ee90fb3f99dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:10:06.742944 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa60e14-b7f3-4e14-a2d6-190a57e46231 service glance] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:06.743223 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa60e14-b7f3-4e14-a2d6-190a57e46231 service glance] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:06.743526 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa60e14-b7f3-4e14-a2d6-190a57e46231 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:06.754912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa60e14-b7f3-4e14-a2d6-190a57e46231 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:10:06.756279 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa60e14-b7f3-4e14-a2d6-190a57e46231 service glance] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:10:06.768165 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1225/2419] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:10:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:10:06.842422 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f13ab-3f46-434c-87a8-add5f31bc0c8 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:06.842775 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f13ab-3f46-434c-87a8-add5f31bc0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:06.843031 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f13ab-3f46-434c-87a8-add5f31bc0c8 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:06.844445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1195/2420] 10.222.0.13 () {54 vars in 921 bytes} [Sat Jun 25 16:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:10:06.849514 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ccbbf-a408-4760-a8fb-17d9e19f01f6 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:06.849806 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ccbbf-a408-4760-a8fb-17d9e19f01f6 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:06.850082 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ccbbf-a408-4760-a8fb-17d9e19f01f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:06.871649 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-c79ccbbf-a408-4760-a8fb-17d9e19f01f6 None None] MFA Rules not processed for user `aaffb9f0bf714822829455c0487376ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:10:06.872900 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c79ccbbf-a408-4760-a8fb-17d9e19f01f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:10:06.932862 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1226/2421] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Jun 25 16:10:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:10:06.946861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f13ab-3f46-434c-87a8-add5f31bc0c8 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:10:06.948867 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ed9f13ab-3f46-434c-87a8-add5f31bc0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:10:06.965860 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f13ab-3f46-434c-87a8-add5f31bc0c8 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:10:06.970382 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab837a-f7f8-4b26-8fda-3b2e465ae173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:10:06.971157 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab837a-f7f8-4b26-8fda-3b2e465ae173 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:06.971508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab837a-f7f8-4b26-8fda-3b2e465ae173 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:06.971851 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab837a-f7f8-4b26-8fda-3b2e465ae173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:06.979619 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab837a-f7f8-4b26-8fda-3b2e465ae173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:10:06.981202 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab837a-f7f8-4b26-8fda-3b2e465ae173 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:10:06.990023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1196/2422] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:10:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:10:07.078860 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536dc554-7b0b-4b60-a49e-4e298410a74e None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:07.079145 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536dc554-7b0b-4b60-a49e-4e298410a74e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:07.079468 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536dc554-7b0b-4b60-a49e-4e298410a74e None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:07.081185 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1227/2423] 10.222.0.13 () {54 vars in 921 bytes} [Sat Jun 25 16:10:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:10:07.086709 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc913dc8-b64b-4782-b46e-1492be82078c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:07.086999 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc913dc8-b64b-4782-b46e-1492be82078c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:07.087304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc913dc8-b64b-4782-b46e-1492be82078c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:07.107195 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-cc913dc8-b64b-4782-b46e-1492be82078c None None] MFA Rules not processed for user `3622c5ccb7534e7c8e8342b4f6643bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:10:07.108387 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-cc913dc8-b64b-4782-b46e-1492be82078c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:10:07.120125 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1197/2424] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Jun 25 16:10:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:10:07.131813 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536dc554-7b0b-4b60-a49e-4e298410a74e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:10:07.144634 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536dc554-7b0b-4b60-a49e-4e298410a74e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:10:07.148810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ab465-0dea-41cb-8afa-e6f4836b9371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:10:07.149474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ab465-0dea-41cb-8afa-e6f4836b9371 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:07.149753 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ab465-0dea-41cb-8afa-e6f4836b9371 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:07.150022 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ab465-0dea-41cb-8afa-e6f4836b9371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:07.163083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ab465-0dea-41cb-8afa-e6f4836b9371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:10:07.164531 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ab465-0dea-41cb-8afa-e6f4836b9371 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:10:07.176818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1228/2425] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:10:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:10:10.082197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69d699-5cf3-4464-a6db-d89ad23a180a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:10.082677 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69d699-5cf3-4464-a6db-d89ad23a180a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:10.082771 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69d699-5cf3-4464-a6db-d89ad23a180a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:10.102461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-4a69d699-5cf3-4464-a6db-d89ad23a180a None None] MFA Rules not processed for user `28b3e73439e64050b406a9f3e9b0160a`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:10:10.103679 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4a69d699-5cf3-4464-a6db-d89ad23a180a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:10:10.166677 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1198/2426] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Jun 25 16:10:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:10:10.179898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ab465-0dea-41cb-8afa-e6f4836b9371 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:10:10.181378 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-064ab465-0dea-41cb-8afa-e6f4836b9371 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:10:10.271934 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ab465-0dea-41cb-8afa-e6f4836b9371 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:10:10.276961 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47750946-38cb-4f52-a39e-cd1b273e6531 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19788c4239ca44f9a31a7d32086c8947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:10:10.277608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47750946-38cb-4f52-a39e-cd1b273e6531 service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:10.277897 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47750946-38cb-4f52-a39e-cd1b273e6531 service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:10.278194 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47750946-38cb-4f52-a39e-cd1b273e6531 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:10.286446 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47750946-38cb-4f52-a39e-cd1b273e6531 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:10:10.288023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47750946-38cb-4f52-a39e-cd1b273e6531 service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:10:10.291823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:10:10.297412 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1229/2427] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Jun 25 16:10:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 25 16:10:20.159967 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=80 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:10:20.295744 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:10:24.625900 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf8aef-8a92-43d3-8dd9-b71e864218c1 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:24.626422 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf8aef-8a92-43d3-8dd9-b71e864218c1 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:24.626422 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf8aef-8a92-43d3-8dd9-b71e864218c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:24.645840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ffdf8aef-8a92-43d3-8dd9-b71e864218c1 None None] MFA Rules not processed for user `8a3137566b004edb909d4e3c702a7f59`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:10:24.647066 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ffdf8aef-8a92-43d3-8dd9-b71e864218c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:10:24.702569 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1199/2428] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:10:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:10:27.010925 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47750946-38cb-4f52-a39e-cd1b273e6531 service swift] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:10:27.012991 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-47750946-38cb-4f52-a39e-cd1b273e6531 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:10:27.083729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47750946-38cb-4f52-a39e-cd1b273e6531 service swift] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:10:27.089400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85843b6-d7d5-4308-b840-623e14d38885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:10:27.090287 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85843b6-d7d5-4308-b840-623e14d38885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:27.090562 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85843b6-d7d5-4308-b840-623e14d38885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:27.090837 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85843b6-d7d5-4308-b840-623e14d38885 admin admin] PATH_INFO: `/v3/users/bf0332a9562946169ff8a45bf5e6f6f3` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:27.092140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85843b6-d7d5-4308-b840-623e14d38885 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf0332a9562946169ff8a45bf5e6f6f3)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:10:27.094100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85843b6-d7d5-4308-b840-623e14d38885 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:10:27.146365 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c85843b6-d7d5-4308-b840-623e14d38885 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf0332a9562946169ff8a45bf5e6f6f3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:27.153741 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c85843b6-d7d5-4308-b840-623e14d38885 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf0332a9562946169ff8a45bf5e6f6f3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:27.154478 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c85843b6-d7d5-4308-b840-623e14d38885 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf0332a9562946169ff8a45bf5e6f6f3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:27.164851 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c85843b6-d7d5-4308-b840-623e14d38885 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf0332a9562946169ff8a45bf5e6f6f3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:27.166199 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c85843b6-d7d5-4308-b840-623e14d38885 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf0332a9562946169ff8a45bf5e6f6f3'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:27.186851 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1230/2429] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:10:27 2022] DELETE /identity/v3/users/bf0332a9562946169ff8a45bf5e6f6f3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:10:27.376421 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf8aef-8a92-43d3-8dd9-b71e864218c1 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:10:27.416399 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf8aef-8a92-43d3-8dd9-b71e864218c1 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:10:27.487304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3435dfe-e74d-423f-9178-4e02d4114d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:10:27.488207 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3435dfe-e74d-423f-9178-4e02d4114d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:27.488522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3435dfe-e74d-423f-9178-4e02d4114d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:27.488812 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3435dfe-e74d-423f-9178-4e02d4114d8b admin admin] PATH_INFO: `/v3/projects/4e1e0c6b3e544d138a1c7911cf155076` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:27.490006 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3435dfe-e74d-423f-9178-4e02d4114d8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e1e0c6b3e544d138a1c7911cf155076)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:10:27.491927 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3435dfe-e74d-423f-9178-4e02d4114d8b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:10:27.553271 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d3435dfe-e74d-423f-9178-4e02d4114d8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e1e0c6b3e544d138a1c7911cf155076', 'request_id': 'req-d3435dfe-e74d-423f-9178-4e02d4114d8b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:27.559490 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d3435dfe-e74d-423f-9178-4e02d4114d8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e1e0c6b3e544d138a1c7911cf155076', 'request_id': 'req-d3435dfe-e74d-423f-9178-4e02d4114d8b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:27.566221 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d3435dfe-e74d-423f-9178-4e02d4114d8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e1e0c6b3e544d138a1c7911cf155076', 'request_id': 'req-d3435dfe-e74d-423f-9178-4e02d4114d8b'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:27.579338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d3435dfe-e74d-423f-9178-4e02d4114d8b admin admin] The token cache is being invalidate because project 4e1e0c6b3e544d138a1c7911cf155076 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:10:27.579768 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d3435dfe-e74d-423f-9178-4e02d4114d8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:27.580424 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d3435dfe-e74d-423f-9178-4e02d4114d8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:27.582043 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1200/2430] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:10:27 2022] DELETE /identity/v3/projects/4e1e0c6b3e544d138a1c7911cf155076 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:10:31.568021 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1422d-2c30-4a54-beea-dccc430b7c0a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:31.568555 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1422d-2c30-4a54-beea-dccc430b7c0a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:31.568653 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1422d-2c30-4a54-beea-dccc430b7c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:31.587924 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-4df1422d-2c30-4a54-beea-dccc430b7c0a None None] MFA Rules not processed for user `28b3e73439e64050b406a9f3e9b0160a`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:10:31.589167 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4df1422d-2c30-4a54-beea-dccc430b7c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:10:31.648372 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1231/2431] 10.222.0.13 () {58 vars in 967 bytes} [Sat Jun 25 16:10:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:10:31.675179 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3435dfe-e74d-423f-9178-4e02d4114d8b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:10:31.677423 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d3435dfe-e74d-423f-9178-4e02d4114d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:10:31.733810 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:10:31.780483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3435dfe-e74d-423f-9178-4e02d4114d8b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:10:31.786863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bdf90-1910-4128-8c17-f605eb560a39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19788c4239ca44f9a31a7d32086c8947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:10:31.787684 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bdf90-1910-4128-8c17-f605eb560a39 service swift] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:31.787981 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bdf90-1910-4128-8c17-f605eb560a39 service swift] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:31.788253 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bdf90-1910-4128-8c17-f605eb560a39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:31.798744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78bdf90-1910-4128-8c17-f605eb560a39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:10:31.800590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78bdf90-1910-4128-8c17-f605eb560a39 service swift] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:10:31.811128 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1201/2432] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Jun 25 16:10:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 25 16:10:36.884707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df1422d-2c30-4a54-beea-dccc430b7c0a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:10:36.886341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4df1422d-2c30-4a54-beea-dccc430b7c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:10:36.915296 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:10:36.960134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df1422d-2c30-4a54-beea-dccc430b7c0a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:10:36.965922 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082eed7-b1f9-45ee-b53a-f92325eec9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:10:36.966792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082eed7-b1f9-45ee-b53a-f92325eec9be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:36.967068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082eed7-b1f9-45ee-b53a-f92325eec9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:36.967339 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082eed7-b1f9-45ee-b53a-f92325eec9be admin admin] PATH_INFO: `/v3/users/b7214c24b8fb4bf9b7804b15114b938b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:36.968613 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082eed7-b1f9-45ee-b53a-f92325eec9be admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7214c24b8fb4bf9b7804b15114b938b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:10:36.970902 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082eed7-b1f9-45ee-b53a-f92325eec9be admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:10:37.024703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4082eed7-b1f9-45ee-b53a-f92325eec9be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7214c24b8fb4bf9b7804b15114b938b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:37.032745 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4082eed7-b1f9-45ee-b53a-f92325eec9be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7214c24b8fb4bf9b7804b15114b938b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:37.033431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4082eed7-b1f9-45ee-b53a-f92325eec9be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7214c24b8fb4bf9b7804b15114b938b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:37.042542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4082eed7-b1f9-45ee-b53a-f92325eec9be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7214c24b8fb4bf9b7804b15114b938b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:37.043154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4082eed7-b1f9-45ee-b53a-f92325eec9be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7214c24b8fb4bf9b7804b15114b938b'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:37.061594 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1232/2433] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:10:36 2022] DELETE /identity/v3/users/b7214c24b8fb4bf9b7804b15114b938b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:10:37.066336 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa339dc-a8e8-4452-8dff-2cb3d997971b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:37.066925 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa339dc-a8e8-4452-8dff-2cb3d997971b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:37.066925 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa339dc-a8e8-4452-8dff-2cb3d997971b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:37.085452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-eaa339dc-a8e8-4452-8dff-2cb3d997971b None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:10:37.086690 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-eaa339dc-a8e8-4452-8dff-2cb3d997971b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:10:37.150754 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1202/2434] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:10:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:10:37.160872 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082eed7-b1f9-45ee-b53a-f92325eec9be admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:10:37.162366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4082eed7-b1f9-45ee-b53a-f92325eec9be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:10:37.221706 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082eed7-b1f9-45ee-b53a-f92325eec9be admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:10:37.226283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99686aed-7bee-43ad-9115-35ad2ec52228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:10:37.226934 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99686aed-7bee-43ad-9115-35ad2ec52228 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:37.227219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99686aed-7bee-43ad-9115-35ad2ec52228 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:37.227497 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99686aed-7bee-43ad-9115-35ad2ec52228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:37.238194 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99686aed-7bee-43ad-9115-35ad2ec52228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:10:37.239621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99686aed-7bee-43ad-9115-35ad2ec52228 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:10:37.249200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1233/2435] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:10:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:10:37.528973 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa339dc-a8e8-4452-8dff-2cb3d997971b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:10:37.530496 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-eaa339dc-a8e8-4452-8dff-2cb3d997971b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:10:37.543928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa339dc-a8e8-4452-8dff-2cb3d997971b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:10:37.549748 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319aa1f7-5599-424e-b4b3-d82e782f3c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:10:37.550629 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319aa1f7-5599-424e-b4b3-d82e782f3c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:10:37.550907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319aa1f7-5599-424e-b4b3-d82e782f3c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:10:37.551183 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319aa1f7-5599-424e-b4b3-d82e782f3c55 admin admin] PATH_INFO: `/v3/projects/c18fde29935c4e3e92999ffa80fce238` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:10:37.552642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319aa1f7-5599-424e-b4b3-d82e782f3c55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c18fde29935c4e3e92999ffa80fce238)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:10:37.555288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319aa1f7-5599-424e-b4b3-d82e782f3c55 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:10:37.608315 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-319aa1f7-5599-424e-b4b3-d82e782f3c55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c18fde29935c4e3e92999ffa80fce238', 'request_id': 'req-319aa1f7-5599-424e-b4b3-d82e782f3c55'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:37.615035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-319aa1f7-5599-424e-b4b3-d82e782f3c55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c18fde29935c4e3e92999ffa80fce238', 'request_id': 'req-319aa1f7-5599-424e-b4b3-d82e782f3c55'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:37.622730 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-319aa1f7-5599-424e-b4b3-d82e782f3c55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c18fde29935c4e3e92999ffa80fce238', 'request_id': 'req-319aa1f7-5599-424e-b4b3-d82e782f3c55'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:37.635801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-319aa1f7-5599-424e-b4b3-d82e782f3c55 admin admin] The token cache is being invalidate because project c18fde29935c4e3e92999ffa80fce238 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:10:37.636096 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-319aa1f7-5599-424e-b4b3-d82e782f3c55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:37.636752 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-319aa1f7-5599-424e-b4b3-d82e782f3c55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:10:37.638326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1203/2436] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:10:37 2022] DELETE /identity/v3/projects/c18fde29935c4e3e92999ffa80fce238 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:10:47.245338 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=68,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:10:47.632956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:11:02.961733 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597d6a0-7f89-4a15-a725-7109eb00c487 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:02.962296 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597d6a0-7f89-4a15-a725-7109eb00c487 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:02.962408 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597d6a0-7f89-4a15-a725-7109eb00c487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:02.988013 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1597d6a0-7f89-4a15-a725-7109eb00c487 None None] MFA Rules not processed for user `19140330ae664e45b85f47f321496191`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:11:02.989263 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1597d6a0-7f89-4a15-a725-7109eb00c487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:03.050823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1234/2437] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:11:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:11:03.087602 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319aa1f7-5599-424e-b4b3-d82e782f3c55 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:03.089314 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-319aa1f7-5599-424e-b4b3-d82e782f3c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:03.149272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319aa1f7-5599-424e-b4b3-d82e782f3c55 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:03.153082 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff42be-4bcb-4dea-8a6e-c59846afd44a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cba086728824ec68fb0ee90fb3f99dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:03.153719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff42be-4bcb-4dea-8a6e-c59846afd44a service glance] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:03.154001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff42be-4bcb-4dea-8a6e-c59846afd44a service glance] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:03.154288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff42be-4bcb-4dea-8a6e-c59846afd44a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:03.156209 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b4ff42be-4bcb-4dea-8a6e-c59846afd44a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:03.241666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff42be-4bcb-4dea-8a6e-c59846afd44a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:03.243103 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff42be-4bcb-4dea-8a6e-c59846afd44a service glance] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:03.254604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1204/2438] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:11:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:11:07.250152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597d6a0-7f89-4a15-a725-7109eb00c487 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:07.252918 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1597d6a0-7f89-4a15-a725-7109eb00c487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:07.321223 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597d6a0-7f89-4a15-a725-7109eb00c487 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:07.325851 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef10da-678d-407a-b46f-b692b5585b01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:07.326506 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef10da-678d-407a-b46f-b692b5585b01 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:07.326783 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef10da-678d-407a-b46f-b692b5585b01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:07.327038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef10da-678d-407a-b46f-b692b5585b01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:07.329036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e3ef10da-678d-407a-b46f-b692b5585b01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:07.342931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef10da-678d-407a-b46f-b692b5585b01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:07.344378 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef10da-678d-407a-b46f-b692b5585b01 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:07.355372 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1235/2439] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:11:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:11:10.373357 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff42be-4bcb-4dea-8a6e-c59846afd44a service glance] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:10.375194 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b4ff42be-4bcb-4dea-8a6e-c59846afd44a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:10.390560 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff42be-4bcb-4dea-8a6e-c59846afd44a service glance] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:10.397766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c650cf-94c6-4c61-a49e-6b3ac06698cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:10.398413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c650cf-94c6-4c61-a49e-6b3ac06698cb service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:10.398688 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c650cf-94c6-4c61-a49e-6b3ac06698cb service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:10.398960 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c650cf-94c6-4c61-a49e-6b3ac06698cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:10.401185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a1c650cf-94c6-4c61-a49e-6b3ac06698cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:10.461874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c650cf-94c6-4c61-a49e-6b3ac06698cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:10.463468 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c650cf-94c6-4c61-a49e-6b3ac06698cb service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:10.472795 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1205/2440] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:11:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:11:15.766986 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef10da-678d-407a-b46f-b692b5585b01 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:15.768947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e3ef10da-678d-407a-b46f-b692b5585b01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:15.785419 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef10da-678d-407a-b46f-b692b5585b01 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:15.804680 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2068d-7701-46d9-b290-22cdc54ab7fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:15.805639 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2068d-7701-46d9-b290-22cdc54ab7fe service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:15.806011 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2068d-7701-46d9-b290-22cdc54ab7fe service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:15.808680 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2068d-7701-46d9-b290-22cdc54ab7fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:15.810655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1ae2068d-7701-46d9-b290-22cdc54ab7fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:15.822937 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2068d-7701-46d9-b290-22cdc54ab7fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:15.824966 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2068d-7701-46d9-b290-22cdc54ab7fe service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:15.841563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1236/2441] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:11:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:11:20.468301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=82 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:11:21.670525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c49e95-27bf-4437-a3b9-620fd84bc1ff None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:21.671126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c49e95-27bf-4437-a3b9-620fd84bc1ff None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:21.671126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c49e95-27bf-4437-a3b9-620fd84bc1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:21.688447 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b9c49e95-27bf-4437-a3b9-620fd84bc1ff None None] MFA Rules not processed for user `744eb9d4e7084b7e9cae36d5de67f0f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:11:21.689640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b9c49e95-27bf-4437-a3b9-620fd84bc1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:21.749350 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1206/2442] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:11:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:11:22.060901 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2068d-7701-46d9-b290-22cdc54ab7fe service nova] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:22.071189 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2068d-7701-46d9-b290-22cdc54ab7fe service nova] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:22.076559 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0078983-4581-4f16-898b-0f96f120dd9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:22.077231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0078983-4581-4f16-898b-0f96f120dd9c service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:22.077513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0078983-4581-4f16-898b-0f96f120dd9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:22.077783 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0078983-4581-4f16-898b-0f96f120dd9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:22.080047 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f0078983-4581-4f16-898b-0f96f120dd9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:22.090683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0078983-4581-4f16-898b-0f96f120dd9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:22.092118 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0078983-4581-4f16-898b-0f96f120dd9c service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:22.100956 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1237/2443] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:11:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:11:22.472309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c49e95-27bf-4437-a3b9-620fd84bc1ff None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:22.473798 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b9c49e95-27bf-4437-a3b9-620fd84bc1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:22.545099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c49e95-27bf-4437-a3b9-620fd84bc1ff None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:22.550519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a1a70-8ce4-487c-9686-2f30f1156af8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474c468a610140f8a5b18d5a1ed1d18e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:22.551165 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a1a70-8ce4-487c-9686-2f30f1156af8 service placement] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:22.551443 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a1a70-8ce4-487c-9686-2f30f1156af8 service placement] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:22.551698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a1a70-8ce4-487c-9686-2f30f1156af8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:22.553517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-124a1a70-8ce4-487c-9686-2f30f1156af8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:22.564920 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a1a70-8ce4-487c-9686-2f30f1156af8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:22.566494 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a1a70-8ce4-487c-9686-2f30f1156af8 service placement] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:22.576352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1207/2444] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:11:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:11:26.645457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0078983-4581-4f16-898b-0f96f120dd9c service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:26.647020 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f0078983-4581-4f16-898b-0f96f120dd9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:26.658211 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0078983-4581-4f16-898b-0f96f120dd9c service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:26.662789 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca728b-d37e-4e52-a478-821d1dd412fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:26.663450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca728b-d37e-4e52-a478-821d1dd412fe service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:26.663714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca728b-d37e-4e52-a478-821d1dd412fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:26.664200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca728b-d37e-4e52-a478-821d1dd412fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:26.666190 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b9ca728b-d37e-4e52-a478-821d1dd412fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:26.677840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca728b-d37e-4e52-a478-821d1dd412fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:26.679750 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca728b-d37e-4e52-a478-821d1dd412fe service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:26.689543 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1238/2445] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:11:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:11:31.193293 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4dc81-1f1e-42e9-b2da-b3cc92f7f751 None None] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:31.193881 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4dc81-1f1e-42e9-b2da-b3cc92f7f751 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:31.193881 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4dc81-1f1e-42e9-b2da-b3cc92f7f751 None None] PATH_INFO: `/` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:31.195198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1208/2446] 10.222.0.13 () {54 vars in 921 bytes} [Sat Jun 25 16:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:11:31.201129 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a02a74-5789-482c-9c8b-436ef586a79f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:31.201613 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a02a74-5789-482c-9c8b-436ef586a79f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:31.201730 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a02a74-5789-482c-9c8b-436ef586a79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:31.208734 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=70 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:11:31.222038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b3a02a74-5789-482c-9c8b-436ef586a79f None None] MFA Rules not processed for user `aaffb9f0bf714822829455c0487376ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:11:31.223362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b3a02a74-5789-482c-9c8b-436ef586a79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:31.230429 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1239/2447] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Jun 25 16:11:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:11:31.242578 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4dc81-1f1e-42e9-b2da-b3cc92f7f751 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:31.254687 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4dc81-1f1e-42e9-b2da-b3cc92f7f751 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:31.262288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238280e-3ecc-4ea2-9456-3379343d3ce3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:31.262941 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238280e-3ecc-4ea2-9456-3379343d3ce3 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:31.263221 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238280e-3ecc-4ea2-9456-3379343d3ce3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:31.263490 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238280e-3ecc-4ea2-9456-3379343d3ce3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:31.271454 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238280e-3ecc-4ea2-9456-3379343d3ce3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:31.272924 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238280e-3ecc-4ea2-9456-3379343d3ce3 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:31.280959 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1209/2448] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:11:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:11:31.374956 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bed8b-03a3-4a50-8d97-0e258af5047d None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:31.375373 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bed8b-03a3-4a50-8d97-0e258af5047d None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:31.375822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8bed8b-03a3-4a50-8d97-0e258af5047d None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:31.377301 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1240/2449] 10.222.0.13 () {54 vars in 921 bytes} [Sat Jun 25 16:11:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:11:31.383324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313d147-e4f7-4bbf-89c8-7b158f5afd7a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:31.383698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313d147-e4f7-4bbf-89c8-7b158f5afd7a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:31.384079 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313d147-e4f7-4bbf-89c8-7b158f5afd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:31.410091 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d313d147-e4f7-4bbf-89c8-7b158f5afd7a None None] MFA Rules not processed for user `3622c5ccb7534e7c8e8342b4f6643bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:11:31.411224 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d313d147-e4f7-4bbf-89c8-7b158f5afd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:31.422314 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1210/2450] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Jun 25 16:11:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:11:31.433252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8bed8b-03a3-4a50-8d97-0e258af5047d None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:31.445694 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8bed8b-03a3-4a50-8d97-0e258af5047d None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:31.449826 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee74156-65bf-48c2-b776-187d3017a8cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:31.450465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee74156-65bf-48c2-b776-187d3017a8cd service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:31.450743 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee74156-65bf-48c2-b776-187d3017a8cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:31.450992 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee74156-65bf-48c2-b776-187d3017a8cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:31.461456 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee74156-65bf-48c2-b776-187d3017a8cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:31.462879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee74156-65bf-48c2-b776-187d3017a8cd service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:31.473563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1241/2451] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:11:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:11:32.667105 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313d147-e4f7-4bbf-89c8-7b158f5afd7a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:32.677327 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313d147-e4f7-4bbf-89c8-7b158f5afd7a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:32.681833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd7061-5a13-410d-b97f-e9ae2dd83a34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:32.682472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd7061-5a13-410d-b97f-e9ae2dd83a34 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:32.682760 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd7061-5a13-410d-b97f-e9ae2dd83a34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:32.683015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd7061-5a13-410d-b97f-e9ae2dd83a34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:32.685209 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7dcd7061-5a13-410d-b97f-e9ae2dd83a34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:32.752449 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd7061-5a13-410d-b97f-e9ae2dd83a34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:32.753904 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd7061-5a13-410d-b97f-e9ae2dd83a34 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:32.764227 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1211/2452] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:11:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:11:36.515456 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee74156-65bf-48c2-b776-187d3017a8cd service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:36.517166 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5ee74156-65bf-48c2-b776-187d3017a8cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:36.531432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee74156-65bf-48c2-b776-187d3017a8cd service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:36.536360 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d1719-ce2c-4a70-a021-4b1eac29f8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:36.537263 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d1719-ce2c-4a70-a021-4b1eac29f8d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:36.537541 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d1719-ce2c-4a70-a021-4b1eac29f8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:36.537825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d1719-ce2c-4a70-a021-4b1eac29f8d3 admin admin] PATH_INFO: `/v3/users/8a3137566b004edb909d4e3c702a7f59` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:36.539033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d1719-ce2c-4a70-a021-4b1eac29f8d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a3137566b004edb909d4e3c702a7f59)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:36.541020 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d1719-ce2c-4a70-a021-4b1eac29f8d3 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:36.596115 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6b8d1719-ce2c-4a70-a021-4b1eac29f8d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a3137566b004edb909d4e3c702a7f59'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:11:36.603241 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6b8d1719-ce2c-4a70-a021-4b1eac29f8d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a3137566b004edb909d4e3c702a7f59'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:11:36.603901 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6b8d1719-ce2c-4a70-a021-4b1eac29f8d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a3137566b004edb909d4e3c702a7f59'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:11:36.612430 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6b8d1719-ce2c-4a70-a021-4b1eac29f8d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a3137566b004edb909d4e3c702a7f59'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:11:36.613066 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6b8d1719-ce2c-4a70-a021-4b1eac29f8d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a3137566b004edb909d4e3c702a7f59'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:11:36.637380 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1242/2453] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:11:36 2022] DELETE /identity/v3/users/8a3137566b004edb909d4e3c702a7f59 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:11:36.820218 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd7061-5a13-410d-b97f-e9ae2dd83a34 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:36.831421 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd7061-5a13-410d-b97f-e9ae2dd83a34 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:36.837693 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:11:36.897950 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2bf10a-4ac3-4b7d-a88f-4772f2902c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:36.898819 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bf10a-4ac3-4b7d-a88f-4772f2902c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:36.899093 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bf10a-4ac3-4b7d-a88f-4772f2902c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:36.899363 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2bf10a-4ac3-4b7d-a88f-4772f2902c14 admin admin] PATH_INFO: `/v3/projects/9bcee16336834b80a389799f89b5b79a` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:36.900705 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2bf10a-4ac3-4b7d-a88f-4772f2902c14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bcee16336834b80a389799f89b5b79a)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:36.902630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2bf10a-4ac3-4b7d-a88f-4772f2902c14 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:36.952389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ce2bf10a-4ac3-4b7d-a88f-4772f2902c14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bcee16336834b80a389799f89b5b79a', 'request_id': 'req-ce2bf10a-4ac3-4b7d-a88f-4772f2902c14'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:11:36.957986 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ce2bf10a-4ac3-4b7d-a88f-4772f2902c14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bcee16336834b80a389799f89b5b79a', 'request_id': 'req-ce2bf10a-4ac3-4b7d-a88f-4772f2902c14'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:11:36.964028 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ce2bf10a-4ac3-4b7d-a88f-4772f2902c14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bcee16336834b80a389799f89b5b79a', 'request_id': 'req-ce2bf10a-4ac3-4b7d-a88f-4772f2902c14'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:11:36.977606 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ce2bf10a-4ac3-4b7d-a88f-4772f2902c14 admin admin] The token cache is being invalidate because project 9bcee16336834b80a389799f89b5b79a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:11:36.977889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ce2bf10a-4ac3-4b7d-a88f-4772f2902c14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:11:36.978542 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ce2bf10a-4ac3-4b7d-a88f-4772f2902c14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:11:36.980088 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1212/2454] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:11:36 2022] DELETE /identity/v3/projects/9bcee16336834b80a389799f89b5b79a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:11:36.985235 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db711b-26cb-43e0-beb6-52232e1b7c6f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:36.985526 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db711b-26cb-43e0-beb6-52232e1b7c6f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:36.985804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db711b-26cb-43e0-beb6-52232e1b7c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:37.003927 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-60db711b-26cb-43e0-beb6-52232e1b7c6f None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:11:37.005190 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-60db711b-26cb-43e0-beb6-52232e1b7c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:37.066746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1243/2455] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:11:37.074528 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414daf58-6841-4b6a-96a9-e7fbf36370f0 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:37.074815 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414daf58-6841-4b6a-96a9-e7fbf36370f0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:37.075096 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414daf58-6841-4b6a-96a9-e7fbf36370f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:37.095112 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-414daf58-6841-4b6a-96a9-e7fbf36370f0 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:11:37.096328 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-414daf58-6841-4b6a-96a9-e7fbf36370f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:37.105532 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1213/2456] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:11:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:11:37.110381 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60db711b-26cb-43e0-beb6-52232e1b7c6f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:37.120329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60db711b-26cb-43e0-beb6-52232e1b7c6f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:37.125334 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df8366-9b5d-40f6-9657-1ef29b10b668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:37.126045 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df8366-9b5d-40f6-9657-1ef29b10b668 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:37.126358 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df8366-9b5d-40f6-9657-1ef29b10b668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:37.126634 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df8366-9b5d-40f6-9657-1ef29b10b668 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:37.127349 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df8366-9b5d-40f6-9657-1ef29b10b668 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:11:37.127656 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df8366-9b5d-40f6-9657-1ef29b10b668 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:37.129291 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df8366-9b5d-40f6-9657-1ef29b10b668 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:37.147763 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1244/2457] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 25 16:11:37.148655 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414daf58-6841-4b6a-96a9-e7fbf36370f0 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:37.159036 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414daf58-6841-4b6a-96a9-e7fbf36370f0 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:37.164279 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603cf9c7-baeb-4018-b315-4368bf08fbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:37.164975 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cf9c7-baeb-4018-b315-4368bf08fbbe admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:37.165250 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cf9c7-baeb-4018-b315-4368bf08fbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:37.165518 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cf9c7-baeb-4018-b315-4368bf08fbbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:37.166213 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603cf9c7-baeb-4018-b315-4368bf08fbbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:37.168158 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603cf9c7-baeb-4018-b315-4368bf08fbbe admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:37.195843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1214/2458] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:11:37 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:11:37.199988 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df8366-9b5d-40f6-9657-1ef29b10b668 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:37.211485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df8366-9b5d-40f6-9657-1ef29b10b668 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:37.251984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:11:37.270457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f2a09-9da8-4569-9fae-5a8f6be64f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:37.271132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f2a09-9da8-4569-9fae-5a8f6be64f5d admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:37.271414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f2a09-9da8-4569-9fae-5a8f6be64f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:37.271687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f2a09-9da8-4569-9fae-5a8f6be64f5d admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:37.272389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f2a09-9da8-4569-9fae-5a8f6be64f5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:37.275602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f2a09-9da8-4569-9fae-5a8f6be64f5d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:37.298869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1245/2459] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:11:37 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:11:37.300070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603cf9c7-baeb-4018-b315-4368bf08fbbe admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:37.311058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603cf9c7-baeb-4018-b315-4368bf08fbbe admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:37.316254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7290bd-916d-4b04-a05d-9e53d86114d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:37.317091 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7290bd-916d-4b04-a05d-9e53d86114d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:37.317379 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7290bd-916d-4b04-a05d-9e53d86114d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:37.317764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7290bd-916d-4b04-a05d-9e53d86114d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:37.318568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7290bd-916d-4b04-a05d-9e53d86114d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:37.320250 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7290bd-916d-4b04-a05d-9e53d86114d1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:37.331368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1215/2460] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:11:37.335681 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f2a09-9da8-4569-9fae-5a8f6be64f5d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:37.346044 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f2a09-9da8-4569-9fae-5a8f6be64f5d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:37.351204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c69c5c-bed2-48f5-a2d5-0e0b5dd8f7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:37.351871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c69c5c-bed2-48f5-a2d5-0e0b5dd8f7ea admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:37.352156 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c69c5c-bed2-48f5-a2d5-0e0b5dd8f7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:37.352640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c69c5c-bed2-48f5-a2d5-0e0b5dd8f7ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:37.353287 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c69c5c-bed2-48f5-a2d5-0e0b5dd8f7ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:37.354865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c69c5c-bed2-48f5-a2d5-0e0b5dd8f7ea admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:37.365935 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1246/2461] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:11:37.370452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7290bd-916d-4b04-a05d-9e53d86114d1 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:37.380740 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7290bd-916d-4b04-a05d-9e53d86114d1 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:37.386359 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54f6d4-b772-4d2c-9c23-06d75fac3af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:37.387249 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54f6d4-b772-4d2c-9c23-06d75fac3af1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:37.387525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54f6d4-b772-4d2c-9c23-06d75fac3af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:37.387826 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54f6d4-b772-4d2c-9c23-06d75fac3af1 admin admin] PATH_INFO: `/v3/projects/93ca9301bea041ce881109c706904a0a/users/5c2141524dc546259cd993ace59e33f3/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:37.403134 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54f6d4-b772-4d2c-9c23-06d75fac3af1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ca9301bea041ce881109c706904a0a, user_id=5c2141524dc546259cd993ace59e33f3, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:37.405972 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54f6d4-b772-4d2c-9c23-06d75fac3af1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:37.420354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1216/2462] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:11:37 2022] PUT /identity/v3/projects/93ca9301bea041ce881109c706904a0a/users/5c2141524dc546259cd993ace59e33f3/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:11:37.425545 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e1072-07e2-4ece-8a40-82859af1384b None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:37.425832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e1072-07e2-4ece-8a40-82859af1384b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:37.426113 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e1072-07e2-4ece-8a40-82859af1384b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:37.443826 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-ee3e1072-07e2-4ece-8a40-82859af1384b None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:11:37.445054 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ee3e1072-07e2-4ece-8a40-82859af1384b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:37.505768 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1247/2463] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:11:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:11:37.516142 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54f6d4-b772-4d2c-9c23-06d75fac3af1 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:37.518106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6b54f6d4-b772-4d2c-9c23-06d75fac3af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:37.560132 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:11:37.579669 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54f6d4-b772-4d2c-9c23-06d75fac3af1 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:37.583816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7634a4-5de6-4bbf-b0f5-ea882721d04b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:37.584469 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7634a4-5de6-4bbf-b0f5-ea882721d04b service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:37.584755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7634a4-5de6-4bbf-b0f5-ea882721d04b service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:37.585026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7634a4-5de6-4bbf-b0f5-ea882721d04b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:37.595208 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7634a4-5de6-4bbf-b0f5-ea882721d04b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:37.596703 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7634a4-5de6-4bbf-b0f5-ea882721d04b service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:37.606786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1217/2464] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:11:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:11:40.016704 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e1072-07e2-4ece-8a40-82859af1384b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:40.018362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-ee3e1072-07e2-4ece-8a40-82859af1384b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:40.164002 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e1072-07e2-4ece-8a40-82859af1384b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:40.184685 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309db60-f244-412b-8d85-f89564cdc86c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:40.187625 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309db60-f244-412b-8d85-f89564cdc86c service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:40.188269 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309db60-f244-412b-8d85-f89564cdc86c service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:40.188718 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d309db60-f244-412b-8d85-f89564cdc86c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:40.193009 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d309db60-f244-412b-8d85-f89564cdc86c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:40.220401 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309db60-f244-412b-8d85-f89564cdc86c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:40.222353 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d309db60-f244-412b-8d85-f89564cdc86c service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:40.248124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1248/2465] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:11:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:11:42.077362 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006edd7d-95ce-41a6-af2f-51a2203efb9e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:42.078198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006edd7d-95ce-41a6-af2f-51a2203efb9e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:42.078198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006edd7d-95ce-41a6-af2f-51a2203efb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:42.104876 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-006edd7d-95ce-41a6-af2f-51a2203efb9e None None] MFA Rules not processed for user `5c2141524dc546259cd993ace59e33f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:11:42.106015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-006edd7d-95ce-41a6-af2f-51a2203efb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:42.188232 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1218/2466] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:11:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:11:42.196001 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309db60-f244-412b-8d85-f89564cdc86c service nova] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:42.197729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d309db60-f244-412b-8d85-f89564cdc86c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:42.214174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d309db60-f244-412b-8d85-f89564cdc86c service nova] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:42.220807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de499523-cf8e-4ba6-8161-f2a0e339cba1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:42.221490 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de499523-cf8e-4ba6-8161-f2a0e339cba1 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:42.221770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de499523-cf8e-4ba6-8161-f2a0e339cba1 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:42.222045 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de499523-cf8e-4ba6-8161-f2a0e339cba1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:42.230910 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de499523-cf8e-4ba6-8161-f2a0e339cba1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:42.232544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de499523-cf8e-4ba6-8161-f2a0e339cba1 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:42.242244 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1249/2467] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:11:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:11:43.632096 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0222b-0004-4efc-b8dd-b1ba7b0437fd None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:43.632633 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0222b-0004-4efc-b8dd-b1ba7b0437fd None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:43.632755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0222b-0004-4efc-b8dd-b1ba7b0437fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:43.652916 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-4ef0222b-0004-4efc-b8dd-b1ba7b0437fd None None] MFA Rules not processed for user `5c2141524dc546259cd993ace59e33f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:11:43.654135 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4ef0222b-0004-4efc-b8dd-b1ba7b0437fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:43.661480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1219/2468] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:11:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:11:47.834977 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de499523-cf8e-4ba6-8161-f2a0e339cba1 service nova] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:11:47.849379 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de499523-cf8e-4ba6-8161-f2a0e339cba1 service nova] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:11:47.856434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0a816-73ee-4b42-b1a2-378f4ac4b5b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:11:47.857354 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0a816-73ee-4b42-b1a2-378f4ac4b5b0 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:11:47.857770 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0a816-73ee-4b42-b1a2-378f4ac4b5b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:11:47.858320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0a816-73ee-4b42-b1a2-378f4ac4b5b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:11:47.863766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a3d0a816-73ee-4b42-b1a2-378f4ac4b5b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:11:47.897426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0a816-73ee-4b42-b1a2-378f4ac4b5b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:11:47.899933 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0a816-73ee-4b42-b1a2-378f4ac4b5b0 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:11:47.913896 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1250/2469] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:11:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:11:53.645964 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=45 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:11:57.907878 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:11.163492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0222b-0004-4efc-b8dd-b1ba7b0437fd None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:11.174140 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0222b-0004-4efc-b8dd-b1ba7b0437fd None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:11.178657 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d5bf9-320a-4828-9355-8f4fc29d43c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:11.179295 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d5bf9-320a-4828-9355-8f4fc29d43c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:11.179573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d5bf9-320a-4828-9355-8f4fc29d43c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:11.179845 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d5bf9-320a-4828-9355-8f4fc29d43c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:11.181934 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-179d5bf9-320a-4828-9355-8f4fc29d43c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:11.194899 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d5bf9-320a-4828-9355-8f4fc29d43c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:11.196330 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d5bf9-320a-4828-9355-8f4fc29d43c4 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:11.206365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1220/2470] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:12:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:12:14.604036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0a816-73ee-4b42-b1a2-378f4ac4b5b0 service nova] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:14.605830 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a3d0a816-73ee-4b42-b1a2-378f4ac4b5b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:14.624790 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0a816-73ee-4b42-b1a2-378f4ac4b5b0 service nova] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:14.631065 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e090a07-95e6-447a-852d-0ec4f384532c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:14.631410 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e090a07-95e6-447a-852d-0ec4f384532c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:14.631695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e090a07-95e6-447a-852d-0ec4f384532c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:14.631969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e090a07-95e6-447a-852d-0ec4f384532c admin admin] PATH_INFO: `/v3/users/19140330ae664e45b85f47f321496191` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:14.633346 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e090a07-95e6-447a-852d-0ec4f384532c admin admin] RBAC: Authorizing `identity:delete_user(user_id=19140330ae664e45b85f47f321496191)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:14.635266 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e090a07-95e6-447a-852d-0ec4f384532c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:14.689939 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9e090a07-95e6-447a-852d-0ec4f384532c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19140330ae664e45b85f47f321496191'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:14.696864 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9e090a07-95e6-447a-852d-0ec4f384532c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19140330ae664e45b85f47f321496191'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:14.697553 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9e090a07-95e6-447a-852d-0ec4f384532c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19140330ae664e45b85f47f321496191'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:14.706096 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9e090a07-95e6-447a-852d-0ec4f384532c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19140330ae664e45b85f47f321496191'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:14.706788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9e090a07-95e6-447a-852d-0ec4f384532c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19140330ae664e45b85f47f321496191'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:14.724841 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1251/2471] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:12:14 2022] DELETE /identity/v3/users/19140330ae664e45b85f47f321496191 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:12:14.910126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d5bf9-320a-4828-9355-8f4fc29d43c4 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:14.924937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d5bf9-320a-4828-9355-8f4fc29d43c4 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:14.984715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251b713-fc75-4205-adf9-ab3bf08f46e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:14.985568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251b713-fc75-4205-adf9-ab3bf08f46e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:14.985843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251b713-fc75-4205-adf9-ab3bf08f46e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:14.986108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251b713-fc75-4205-adf9-ab3bf08f46e2 admin admin] PATH_INFO: `/v3/projects/b1bba8f74dbd433bbffdb2ffbfc481fb` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:14.987341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251b713-fc75-4205-adf9-ab3bf08f46e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1bba8f74dbd433bbffdb2ffbfc481fb)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:14.989298 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251b713-fc75-4205-adf9-ab3bf08f46e2 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:15.051254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9251b713-fc75-4205-adf9-ab3bf08f46e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1bba8f74dbd433bbffdb2ffbfc481fb', 'request_id': 'req-9251b713-fc75-4205-adf9-ab3bf08f46e2'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:15.056865 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9251b713-fc75-4205-adf9-ab3bf08f46e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1bba8f74dbd433bbffdb2ffbfc481fb', 'request_id': 'req-9251b713-fc75-4205-adf9-ab3bf08f46e2'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:15.063731 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9251b713-fc75-4205-adf9-ab3bf08f46e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1bba8f74dbd433bbffdb2ffbfc481fb', 'request_id': 'req-9251b713-fc75-4205-adf9-ab3bf08f46e2'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:15.077968 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9251b713-fc75-4205-adf9-ab3bf08f46e2 admin admin] The token cache is being invalidate because project b1bba8f74dbd433bbffdb2ffbfc481fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:12:15.078179 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9251b713-fc75-4205-adf9-ab3bf08f46e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:15.078833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9251b713-fc75-4205-adf9-ab3bf08f46e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:15.080386 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1221/2472] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:12:14 2022] DELETE /identity/v3/projects/b1bba8f74dbd433bbffdb2ffbfc481fb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:12:15.085445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532398cc-832b-4f5a-bdaf-cfba7d9960d1 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:15.085733 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532398cc-832b-4f5a-bdaf-cfba7d9960d1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:15.086007 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532398cc-832b-4f5a-bdaf-cfba7d9960d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:15.104277 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-532398cc-832b-4f5a-bdaf-cfba7d9960d1 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:15.105530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-532398cc-832b-4f5a-bdaf-cfba7d9960d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:15.171687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1252/2473] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:15.179590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388aff4e-2974-43ca-92ad-81c6f57794ff None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:15.179873 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388aff4e-2974-43ca-92ad-81c6f57794ff None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:15.180162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388aff4e-2974-43ca-92ad-81c6f57794ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:15.199285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-388aff4e-2974-43ca-92ad-81c6f57794ff None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:15.200550 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-388aff4e-2974-43ca-92ad-81c6f57794ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:15.209261 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1222/2474] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:15.213521 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532398cc-832b-4f5a-bdaf-cfba7d9960d1 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:15.224363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532398cc-832b-4f5a-bdaf-cfba7d9960d1 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:15.230742 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e9761-43db-48c5-9f7b-314c3c6e816b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:15.231399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e9761-43db-48c5-9f7b-314c3c6e816b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:15.231678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e9761-43db-48c5-9f7b-314c3c6e816b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:15.231949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e9761-43db-48c5-9f7b-314c3c6e816b admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:15.232727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e9761-43db-48c5-9f7b-314c3c6e816b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:12:15.233074 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e9761-43db-48c5-9f7b-314c3c6e816b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:15.234674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e9761-43db-48c5-9f7b-314c3c6e816b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:15.246676 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1253/2475] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:12:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:12:15.250959 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388aff4e-2974-43ca-92ad-81c6f57794ff None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:15.260211 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388aff4e-2974-43ca-92ad-81c6f57794ff None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:15.265292 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d168b-5094-4439-9ab5-b84abc668f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:15.265931 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d168b-5094-4439-9ab5-b84abc668f1d admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:15.266209 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d168b-5094-4439-9ab5-b84abc668f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:15.266479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d168b-5094-4439-9ab5-b84abc668f1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:15.267176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d168b-5094-4439-9ab5-b84abc668f1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:15.269130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d168b-5094-4439-9ab5-b84abc668f1d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:15.296996 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1223/2476] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:12:15 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:12:15.301171 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e9761-43db-48c5-9f7b-314c3c6e816b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:15.311056 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e9761-43db-48c5-9f7b-314c3c6e816b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:15.371534 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e4f98-9d9f-4b15-ac61-1b843c9d4eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:15.372117 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e4f98-9d9f-4b15-ac61-1b843c9d4eda admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:15.372393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e4f98-9d9f-4b15-ac61-1b843c9d4eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:15.372725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e4f98-9d9f-4b15-ac61-1b843c9d4eda admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:15.373417 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e4f98-9d9f-4b15-ac61-1b843c9d4eda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:15.375441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e4f98-9d9f-4b15-ac61-1b843c9d4eda admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:15.382084 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:15.397998 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1254/2477] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:12:15 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:12:15.402539 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d168b-5094-4439-9ab5-b84abc668f1d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:15.413998 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d168b-5094-4439-9ab5-b84abc668f1d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:15.419854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b137d-7042-4672-bf88-911a6470d46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:15.420569 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b137d-7042-4672-bf88-911a6470d46b admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:15.420851 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b137d-7042-4672-bf88-911a6470d46b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:15.421121 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b137d-7042-4672-bf88-911a6470d46b admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:15.421855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b137d-7042-4672-bf88-911a6470d46b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:15.423469 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b137d-7042-4672-bf88-911a6470d46b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:15.434297 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1224/2478] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:12:15.438398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e4f98-9d9f-4b15-ac61-1b843c9d4eda admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:15.450072 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e4f98-9d9f-4b15-ac61-1b843c9d4eda admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:15.454767 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec929969-9aa4-40ef-b629-7e895f8eb858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:15.455421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec929969-9aa4-40ef-b629-7e895f8eb858 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:15.455702 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec929969-9aa4-40ef-b629-7e895f8eb858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:15.455978 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec929969-9aa4-40ef-b629-7e895f8eb858 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:15.456739 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec929969-9aa4-40ef-b629-7e895f8eb858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:15.458327 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec929969-9aa4-40ef-b629-7e895f8eb858 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:15.468318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1255/2479] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:12:15.472535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b137d-7042-4672-bf88-911a6470d46b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:15.482471 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b137d-7042-4672-bf88-911a6470d46b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:15.487574 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ea477-76a0-47a5-8231-ea97ad380508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:15.488475 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ea477-76a0-47a5-8231-ea97ad380508 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:15.488749 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ea477-76a0-47a5-8231-ea97ad380508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:15.489015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ea477-76a0-47a5-8231-ea97ad380508 admin admin] PATH_INFO: `/v3/projects/9d2980c65c0a4157ae8f9a022a9f5eca/users/76801fe174ac42b6aa7ec0661cd6159d/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:15.503428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ea477-76a0-47a5-8231-ea97ad380508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d2980c65c0a4157ae8f9a022a9f5eca, user_id=76801fe174ac42b6aa7ec0661cd6159d, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:15.506104 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ea477-76a0-47a5-8231-ea97ad380508 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:15.520982 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1225/2480] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:12:15 2022] PUT /identity/v3/projects/9d2980c65c0a4157ae8f9a022a9f5eca/users/76801fe174ac42b6aa7ec0661cd6159d/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:12:15.531535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c36c7-7be0-4b23-a5c1-bb25c34492db None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:15.531822 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c36c7-7be0-4b23-a5c1-bb25c34492db None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:15.532093 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c36c7-7be0-4b23-a5c1-bb25c34492db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:15.560279 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-462c36c7-7be0-4b23-a5c1-bb25c34492db None None] MFA Rules not processed for user `76801fe174ac42b6aa7ec0661cd6159d`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:15.561582 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-462c36c7-7be0-4b23-a5c1-bb25c34492db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:15.626667 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1256/2481] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:15.638761 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ea477-76a0-47a5-8231-ea97ad380508 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:15.642337 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c21ea477-76a0-47a5-8231-ea97ad380508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:15.659836 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:15.722787 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ea477-76a0-47a5-8231-ea97ad380508 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:15.728960 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd384da-1506-4906-b1de-4db9114299d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:15.729610 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd384da-1506-4906-b1de-4db9114299d9 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:15.729889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd384da-1506-4906-b1de-4db9114299d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:15.730167 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd384da-1506-4906-b1de-4db9114299d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:15.738635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd384da-1506-4906-b1de-4db9114299d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:15.740227 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd384da-1506-4906-b1de-4db9114299d9 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:15.749587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1226/2482] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:12:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:12:18.468323 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c36c7-7be0-4b23-a5c1-bb25c34492db None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:18.538696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c36c7-7be0-4b23-a5c1-bb25c34492db None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:18.545190 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645ae16-e767-4cdc-95b1-6ec88906e1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:18.546537 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645ae16-e767-4cdc-95b1-6ec88906e1d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:18.546834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645ae16-e767-4cdc-95b1-6ec88906e1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:18.547109 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645ae16-e767-4cdc-95b1-6ec88906e1d5 admin admin] PATH_INFO: `/v3/users/76801fe174ac42b6aa7ec0661cd6159d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:18.548533 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645ae16-e767-4cdc-95b1-6ec88906e1d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76801fe174ac42b6aa7ec0661cd6159d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:18.550603 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645ae16-e767-4cdc-95b1-6ec88906e1d5 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:18.605051 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6645ae16-e767-4cdc-95b1-6ec88906e1d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76801fe174ac42b6aa7ec0661cd6159d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:18.612539 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6645ae16-e767-4cdc-95b1-6ec88906e1d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76801fe174ac42b6aa7ec0661cd6159d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:18.613287 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6645ae16-e767-4cdc-95b1-6ec88906e1d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76801fe174ac42b6aa7ec0661cd6159d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:18.622092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6645ae16-e767-4cdc-95b1-6ec88906e1d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76801fe174ac42b6aa7ec0661cd6159d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:18.622728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6645ae16-e767-4cdc-95b1-6ec88906e1d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76801fe174ac42b6aa7ec0661cd6159d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:18.628276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:18.644313 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1257/2483] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:12:18 2022] DELETE /identity/v3/users/76801fe174ac42b6aa7ec0661cd6159d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:12:18.649013 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aac259-ad15-4014-8793-47a2d5d8df5b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:18.649541 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aac259-ad15-4014-8793-47a2d5d8df5b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:18.649541 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aac259-ad15-4014-8793-47a2d5d8df5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:18.668090 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-28aac259-ad15-4014-8793-47a2d5d8df5b None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:18.669347 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-28aac259-ad15-4014-8793-47a2d5d8df5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:18.734115 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1227/2484] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:18.744230 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645ae16-e767-4cdc-95b1-6ec88906e1d5 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:18.745834 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6645ae16-e767-4cdc-95b1-6ec88906e1d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:18.812177 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645ae16-e767-4cdc-95b1-6ec88906e1d5 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:18.816561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef770e6-b3e3-4b93-aa61-d4a3c6a759ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:18.817234 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef770e6-b3e3-4b93-aa61-d4a3c6a759ea service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:18.817514 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef770e6-b3e3-4b93-aa61-d4a3c6a759ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:18.817771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef770e6-b3e3-4b93-aa61-d4a3c6a759ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:18.827754 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef770e6-b3e3-4b93-aa61-d4a3c6a759ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:18.829208 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef770e6-b3e3-4b93-aa61-d4a3c6a759ea service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:18.839162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1258/2485] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:12:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:12:19.135086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aac259-ad15-4014-8793-47a2d5d8df5b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:19.137198 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-28aac259-ad15-4014-8793-47a2d5d8df5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:19.150980 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aac259-ad15-4014-8793-47a2d5d8df5b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:19.157297 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a8d69-b09e-40ad-b54d-29bd95bae476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:19.158176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a8d69-b09e-40ad-b54d-29bd95bae476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:19.158457 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a8d69-b09e-40ad-b54d-29bd95bae476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:19.158765 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a8d69-b09e-40ad-b54d-29bd95bae476 admin admin] PATH_INFO: `/v3/projects/9d2980c65c0a4157ae8f9a022a9f5eca` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:19.159981 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2a8d69-b09e-40ad-b54d-29bd95bae476 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d2980c65c0a4157ae8f9a022a9f5eca)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:19.162118 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2a8d69-b09e-40ad-b54d-29bd95bae476 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:19.213497 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0b2a8d69-b09e-40ad-b54d-29bd95bae476 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d2980c65c0a4157ae8f9a022a9f5eca', 'request_id': 'req-0b2a8d69-b09e-40ad-b54d-29bd95bae476'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:19.218967 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0b2a8d69-b09e-40ad-b54d-29bd95bae476 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d2980c65c0a4157ae8f9a022a9f5eca', 'request_id': 'req-0b2a8d69-b09e-40ad-b54d-29bd95bae476'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:19.225062 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0b2a8d69-b09e-40ad-b54d-29bd95bae476 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d2980c65c0a4157ae8f9a022a9f5eca', 'request_id': 'req-0b2a8d69-b09e-40ad-b54d-29bd95bae476'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:19.238052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0b2a8d69-b09e-40ad-b54d-29bd95bae476 admin admin] The token cache is being invalidate because project 9d2980c65c0a4157ae8f9a022a9f5eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:12:19.238341 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0b2a8d69-b09e-40ad-b54d-29bd95bae476 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:19.238958 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0b2a8d69-b09e-40ad-b54d-29bd95bae476 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:19.240559 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1228/2486] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:12:19 2022] DELETE /identity/v3/projects/9d2980c65c0a4157ae8f9a022a9f5eca => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:12:19.245587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e405e-4936-4578-9548-0a95187f07c1 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:19.246056 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e405e-4936-4578-9548-0a95187f07c1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:19.246176 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e405e-4936-4578-9548-0a95187f07c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:19.266410 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8b3e405e-4936-4578-9548-0a95187f07c1 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:19.267632 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8b3e405e-4936-4578-9548-0a95187f07c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:19.333759 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1259/2487] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:19.343058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fcafe-6000-4aad-b7e8-66ec2adf9f55 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:19.343298 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fcafe-6000-4aad-b7e8-66ec2adf9f55 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:19.343566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fcafe-6000-4aad-b7e8-66ec2adf9f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:19.362456 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d60fcafe-6000-4aad-b7e8-66ec2adf9f55 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:19.363718 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d60fcafe-6000-4aad-b7e8-66ec2adf9f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:19.371788 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1229/2488] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:19.376305 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e405e-4936-4578-9548-0a95187f07c1 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:19.386279 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e405e-4936-4578-9548-0a95187f07c1 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:19.391249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0744e-32a8-49ce-9b39-9beb8760d094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:19.391920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0744e-32a8-49ce-9b39-9beb8760d094 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:19.392204 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0744e-32a8-49ce-9b39-9beb8760d094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:19.392516 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0744e-32a8-49ce-9b39-9beb8760d094 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:19.393254 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0744e-32a8-49ce-9b39-9beb8760d094 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:12:19.393569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0744e-32a8-49ce-9b39-9beb8760d094 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:19.395171 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0744e-32a8-49ce-9b39-9beb8760d094 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:19.407859 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1260/2489] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:12:19.412195 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fcafe-6000-4aad-b7e8-66ec2adf9f55 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:19.423245 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fcafe-6000-4aad-b7e8-66ec2adf9f55 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:19.428057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bcf69-fbe2-41af-9995-8a5ef2eb53e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:19.428823 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bcf69-fbe2-41af-9995-8a5ef2eb53e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:19.429164 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bcf69-fbe2-41af-9995-8a5ef2eb53e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:19.429522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bcf69-fbe2-41af-9995-8a5ef2eb53e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:19.430279 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2bcf69-fbe2-41af-9995-8a5ef2eb53e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:19.432320 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2bcf69-fbe2-41af-9995-8a5ef2eb53e8 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:19.446234 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:19.464670 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1230/2490] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:12:19 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:12:19.466002 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0744e-32a8-49ce-9b39-9beb8760d094 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:19.476129 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0744e-32a8-49ce-9b39-9beb8760d094 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:19.536676 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03319eab-bf69-4a06-899c-6eef899edc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:19.537350 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03319eab-bf69-4a06-899c-6eef899edc38 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:19.537628 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03319eab-bf69-4a06-899c-6eef899edc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:19.537902 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03319eab-bf69-4a06-899c-6eef899edc38 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:19.538579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03319eab-bf69-4a06-899c-6eef899edc38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:19.540536 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03319eab-bf69-4a06-899c-6eef899edc38 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:19.560916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1261/2491] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:12:19 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:12:19.565410 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bcf69-fbe2-41af-9995-8a5ef2eb53e8 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:19.576612 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bcf69-fbe2-41af-9995-8a5ef2eb53e8 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:19.582838 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e70834-7194-428f-ab4a-0e309d099729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:19.583509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e70834-7194-428f-ab4a-0e309d099729 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:19.583933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e70834-7194-428f-ab4a-0e309d099729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:19.584214 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e70834-7194-428f-ab4a-0e309d099729 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:19.585008 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e70834-7194-428f-ab4a-0e309d099729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:19.586632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e70834-7194-428f-ab4a-0e309d099729 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:19.597916 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1231/2492] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:12:19.602184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03319eab-bf69-4a06-899c-6eef899edc38 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:19.613388 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03319eab-bf69-4a06-899c-6eef899edc38 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:19.619330 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ae1ed-31c5-4ff7-83aa-d192b7dce335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:19.620011 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ae1ed-31c5-4ff7-83aa-d192b7dce335 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:19.620290 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ae1ed-31c5-4ff7-83aa-d192b7dce335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:19.620595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ae1ed-31c5-4ff7-83aa-d192b7dce335 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:19.621318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ae1ed-31c5-4ff7-83aa-d192b7dce335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:19.622919 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ae1ed-31c5-4ff7-83aa-d192b7dce335 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:19.624187 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:19.635427 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1262/2493] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:12:19.640711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e70834-7194-428f-ab4a-0e309d099729 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:19.651114 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e70834-7194-428f-ab4a-0e309d099729 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:19.656706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a069d-24ee-45c0-ba6c-51a3e2633b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:19.657595 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a069d-24ee-45c0-ba6c-51a3e2633b3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:19.657878 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a069d-24ee-45c0-ba6c-51a3e2633b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:19.658149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a069d-24ee-45c0-ba6c-51a3e2633b3a admin admin] PATH_INFO: `/v3/projects/2eb14c8a58704f8ca56fcb6a0630ec0c/users/88edef803a6140fa834e02015e4d3e15/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:19.673631 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a069d-24ee-45c0-ba6c-51a3e2633b3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eb14c8a58704f8ca56fcb6a0630ec0c, user_id=88edef803a6140fa834e02015e4d3e15, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:19.676296 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a069d-24ee-45c0-ba6c-51a3e2633b3a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:19.691478 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1232/2494] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:12:19 2022] PUT /identity/v3/projects/2eb14c8a58704f8ca56fcb6a0630ec0c/users/88edef803a6140fa834e02015e4d3e15/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:12:19.702019 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82251693-5213-4fb8-a0a3-a82293b16a41 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:19.702445 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82251693-5213-4fb8-a0a3-a82293b16a41 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:19.702707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82251693-5213-4fb8-a0a3-a82293b16a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:19.733972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-82251693-5213-4fb8-a0a3-a82293b16a41 None None] MFA Rules not processed for user `88edef803a6140fa834e02015e4d3e15`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:19.735198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-82251693-5213-4fb8-a0a3-a82293b16a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:19.817344 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1263/2495] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:19 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:19.828338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a069d-24ee-45c0-ba6c-51a3e2633b3a admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:19.830496 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d34a069d-24ee-45c0-ba6c-51a3e2633b3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:19.890835 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a069d-24ee-45c0-ba6c-51a3e2633b3a admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:19.895248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0bc2c-4679-42b2-8c54-fcdbeb52405e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:19.895982 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0bc2c-4679-42b2-8c54-fcdbeb52405e service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:19.896288 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0bc2c-4679-42b2-8c54-fcdbeb52405e service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:19.896640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0bc2c-4679-42b2-8c54-fcdbeb52405e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:19.906026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0bc2c-4679-42b2-8c54-fcdbeb52405e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:19.907461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0bc2c-4679-42b2-8c54-fcdbeb52405e service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:19.916121 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1233/2496] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:12:19 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:12:28.358054 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82251693-5213-4fb8-a0a3-a82293b16a41 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:28.433173 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82251693-5213-4fb8-a0a3-a82293b16a41 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:28.439003 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377cf1e9-a290-465d-ae11-0e2ae1d36033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:28.439901 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cf1e9-a290-465d-ae11-0e2ae1d36033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:28.440229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cf1e9-a290-465d-ae11-0e2ae1d36033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:28.440674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377cf1e9-a290-465d-ae11-0e2ae1d36033 admin admin] PATH_INFO: `/v3/users/88edef803a6140fa834e02015e4d3e15` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:28.442049 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377cf1e9-a290-465d-ae11-0e2ae1d36033 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88edef803a6140fa834e02015e4d3e15)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:28.444228 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377cf1e9-a290-465d-ae11-0e2ae1d36033 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:28.504678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-377cf1e9-a290-465d-ae11-0e2ae1d36033 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88edef803a6140fa834e02015e4d3e15'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:28.513164 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-377cf1e9-a290-465d-ae11-0e2ae1d36033 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88edef803a6140fa834e02015e4d3e15'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:28.513867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-377cf1e9-a290-465d-ae11-0e2ae1d36033 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88edef803a6140fa834e02015e4d3e15'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:28.522591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-377cf1e9-a290-465d-ae11-0e2ae1d36033 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88edef803a6140fa834e02015e4d3e15'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:28.523239 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-377cf1e9-a290-465d-ae11-0e2ae1d36033 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88edef803a6140fa834e02015e4d3e15'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:28.540815 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1264/2497] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:12:28 2022] DELETE /identity/v3/users/88edef803a6140fa834e02015e4d3e15 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:12:28.546104 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b805b8c-1cae-41a7-9f5b-035e38ecffac None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:28.546608 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b805b8c-1cae-41a7-9f5b-035e38ecffac None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:28.546712 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b805b8c-1cae-41a7-9f5b-035e38ecffac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:28.564431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-3b805b8c-1cae-41a7-9f5b-035e38ecffac None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:28.565653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3b805b8c-1cae-41a7-9f5b-035e38ecffac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:28.626972 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1234/2498] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:12:28.636805 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377cf1e9-a290-465d-ae11-0e2ae1d36033 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:28.638427 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-377cf1e9-a290-465d-ae11-0e2ae1d36033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:28.646639 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:28.712477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377cf1e9-a290-465d-ae11-0e2ae1d36033 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:28.717174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c903ec-d58a-4b6b-8aba-37e530bf88c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:28.717856 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c903ec-d58a-4b6b-8aba-37e530bf88c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:28.718173 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c903ec-d58a-4b6b-8aba-37e530bf88c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:28.718448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c903ec-d58a-4b6b-8aba-37e530bf88c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:28.729404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c903ec-d58a-4b6b-8aba-37e530bf88c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:28.730875 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c903ec-d58a-4b6b-8aba-37e530bf88c3 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:28.740701 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1265/2499] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:12:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:12:29.040025 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b805b8c-1cae-41a7-9f5b-035e38ecffac None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:29.042006 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3b805b8c-1cae-41a7-9f5b-035e38ecffac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:29.055820 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b805b8c-1cae-41a7-9f5b-035e38ecffac None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:29.061687 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfca870-cdc6-4af9-99f5-fc0be6104cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:29.062586 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfca870-cdc6-4af9-99f5-fc0be6104cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:29.062866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfca870-cdc6-4af9-99f5-fc0be6104cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:29.063137 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfca870-cdc6-4af9-99f5-fc0be6104cd3 admin admin] PATH_INFO: `/v3/projects/2eb14c8a58704f8ca56fcb6a0630ec0c` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:29.065321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfca870-cdc6-4af9-99f5-fc0be6104cd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eb14c8a58704f8ca56fcb6a0630ec0c)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:29.067266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfca870-cdc6-4af9-99f5-fc0be6104cd3 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:29.119784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0cfca870-cdc6-4af9-99f5-fc0be6104cd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eb14c8a58704f8ca56fcb6a0630ec0c', 'request_id': 'req-0cfca870-cdc6-4af9-99f5-fc0be6104cd3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:29.126414 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:INSERT=2,keystone:SELECT=92,keystone:DELETE=6 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:29.128004 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0cfca870-cdc6-4af9-99f5-fc0be6104cd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eb14c8a58704f8ca56fcb6a0630ec0c', 'request_id': 'req-0cfca870-cdc6-4af9-99f5-fc0be6104cd3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:29.136203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0cfca870-cdc6-4af9-99f5-fc0be6104cd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eb14c8a58704f8ca56fcb6a0630ec0c', 'request_id': 'req-0cfca870-cdc6-4af9-99f5-fc0be6104cd3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:29.161001 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0cfca870-cdc6-4af9-99f5-fc0be6104cd3 admin admin] The token cache is being invalidate because project 2eb14c8a58704f8ca56fcb6a0630ec0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:12:29.163386 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0cfca870-cdc6-4af9-99f5-fc0be6104cd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:29.163386 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-0cfca870-cdc6-4af9-99f5-fc0be6104cd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:29.164251 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1235/2500] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:12:29 2022] DELETE /identity/v3/projects/2eb14c8a58704f8ca56fcb6a0630ec0c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:12:29.169495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e2f0c-e9e3-42c8-9e2b-411315f06cd6 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:29.169781 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e2f0c-e9e3-42c8-9e2b-411315f06cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:29.170056 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e2f0c-e9e3-42c8-9e2b-411315f06cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:29.194359 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-cb8e2f0c-e9e3-42c8-9e2b-411315f06cd6 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:29.195570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-cb8e2f0c-e9e3-42c8-9e2b-411315f06cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:29.274481 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1266/2501] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:29.282690 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8adf2a-ea5b-43ab-8347-94881a857c13 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:29.283034 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8adf2a-ea5b-43ab-8347-94881a857c13 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:29.283328 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8adf2a-ea5b-43ab-8347-94881a857c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:29.305833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5f8adf2a-ea5b-43ab-8347-94881a857c13 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:29.307042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5f8adf2a-ea5b-43ab-8347-94881a857c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:29.315323 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1236/2502] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:29.319621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e2f0c-e9e3-42c8-9e2b-411315f06cd6 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:29.330196 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e2f0c-e9e3-42c8-9e2b-411315f06cd6 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:29.336266 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49895ae-a3ef-46d9-8dc7-56da1ae28cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:29.336955 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49895ae-a3ef-46d9-8dc7-56da1ae28cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:29.337232 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49895ae-a3ef-46d9-8dc7-56da1ae28cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:29.337504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49895ae-a3ef-46d9-8dc7-56da1ae28cb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:29.338239 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49895ae-a3ef-46d9-8dc7-56da1ae28cb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:12:29.338542 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49895ae-a3ef-46d9-8dc7-56da1ae28cb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:29.340147 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49895ae-a3ef-46d9-8dc7-56da1ae28cb3 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:29.352843 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1267/2503] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:12:29.357360 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8adf2a-ea5b-43ab-8347-94881a857c13 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:29.367244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8adf2a-ea5b-43ab-8347-94881a857c13 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:29.373572 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e38e8b-b896-419d-88e0-8014f7cc6922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:29.374258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e38e8b-b896-419d-88e0-8014f7cc6922 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:29.374535 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e38e8b-b896-419d-88e0-8014f7cc6922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:29.374834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e38e8b-b896-419d-88e0-8014f7cc6922 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:29.375536 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e38e8b-b896-419d-88e0-8014f7cc6922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:29.377499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e38e8b-b896-419d-88e0-8014f7cc6922 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:29.416657 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1237/2504] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:12:29 2022] POST /identity/v3/projects => generated 369 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:12:29.422239 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49895ae-a3ef-46d9-8dc7-56da1ae28cb3 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:29.432396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49895ae-a3ef-46d9-8dc7-56da1ae28cb3 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:29.498043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed190964-b99c-4f69-bf3c-a000e1f42b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:29.498863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed190964-b99c-4f69-bf3c-a000e1f42b27 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:29.499351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed190964-b99c-4f69-bf3c-a000e1f42b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:29.499486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed190964-b99c-4f69-bf3c-a000e1f42b27 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:29.500162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed190964-b99c-4f69-bf3c-a000e1f42b27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:29.502657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed190964-b99c-4f69-bf3c-a000e1f42b27 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:29.525289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1268/2505] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:12:29 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:12:29.529768 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e38e8b-b896-419d-88e0-8014f7cc6922 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:29.542764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e38e8b-b896-419d-88e0-8014f7cc6922 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:29.550258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21e794-c134-45d9-ae42-4647b87c5744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:29.551150 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21e794-c134-45d9-ae42-4647b87c5744 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:29.551583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21e794-c134-45d9-ae42-4647b87c5744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:29.551859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21e794-c134-45d9-ae42-4647b87c5744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:29.552664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21e794-c134-45d9-ae42-4647b87c5744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:29.554305 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21e794-c134-45d9-ae42-4647b87c5744 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:29.567726 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1238/2506] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:12:29.572474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed190964-b99c-4f69-bf3c-a000e1f42b27 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:29.584701 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed190964-b99c-4f69-bf3c-a000e1f42b27 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:29.590699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a2d20-1dff-4983-899e-9c21c52f0e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:29.591362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a2d20-1dff-4983-899e-9c21c52f0e85 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:29.591644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a2d20-1dff-4983-899e-9c21c52f0e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:29.591922 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a2d20-1dff-4983-899e-9c21c52f0e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:29.593126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a2d20-1dff-4983-899e-9c21c52f0e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:29.594735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a2d20-1dff-4983-899e-9c21c52f0e85 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:29.605797 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1269/2507] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:12:29.610214 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21e794-c134-45d9-ae42-4647b87c5744 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:29.620704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21e794-c134-45d9-ae42-4647b87c5744 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:29.628822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5497d6-a8b9-4a75-a344-9a362280174f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:29.629717 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5497d6-a8b9-4a75-a344-9a362280174f admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:29.629981 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5497d6-a8b9-4a75-a344-9a362280174f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:29.630286 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5497d6-a8b9-4a75-a344-9a362280174f admin admin] PATH_INFO: `/v3/projects/5c418ab33a154076b191667695ec955f/users/db7cd6475b31454293f142137b8e6fe1/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:29.646303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5497d6-a8b9-4a75-a344-9a362280174f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c418ab33a154076b191667695ec955f, user_id=db7cd6475b31454293f142137b8e6fe1, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:29.649022 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5497d6-a8b9-4a75-a344-9a362280174f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:29.669533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1239/2508] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:12:29 2022] PUT /identity/v3/projects/5c418ab33a154076b191667695ec955f/users/db7cd6475b31454293f142137b8e6fe1/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:12:29.683385 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7a59d-f61a-4084-beb9-2de4a3ef3e60 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:29.683671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7a59d-f61a-4084-beb9-2de4a3ef3e60 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:29.683946 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7a59d-f61a-4084-beb9-2de4a3ef3e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:29.709463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:29.713206 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5ab7a59d-f61a-4084-beb9-2de4a3ef3e60 None None] MFA Rules not processed for user `db7cd6475b31454293f142137b8e6fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:29.715415 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5ab7a59d-f61a-4084-beb9-2de4a3ef3e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:29.792336 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1270/2509] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:29.797727 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5497d6-a8b9-4a75-a344-9a362280174f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:29.868143 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5497d6-a8b9-4a75-a344-9a362280174f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:29.874697 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097e1fb-bc8e-4a48-a655-55c105b52b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:29.875366 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097e1fb-bc8e-4a48-a655-55c105b52b4a admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:29.875650 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097e1fb-bc8e-4a48-a655-55c105b52b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:29.875937 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097e1fb-bc8e-4a48-a655-55c105b52b4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:29.876673 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097e1fb-bc8e-4a48-a655-55c105b52b4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:29.878631 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097e1fb-bc8e-4a48-a655-55c105b52b4a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:29.906851 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1240/2510] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:12:29 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:12:29.911245 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7a59d-f61a-4084-beb9-2de4a3ef3e60 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:29.922335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab7a59d-f61a-4084-beb9-2de4a3ef3e60 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:29.980235 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea6e4d-2bb3-4ae2-8a88-927d12a9e41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:29.980943 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea6e4d-2bb3-4ae2-8a88-927d12a9e41b admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:29.981226 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea6e4d-2bb3-4ae2-8a88-927d12a9e41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:29.981511 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea6e4d-2bb3-4ae2-8a88-927d12a9e41b admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:29.982206 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea6e4d-2bb3-4ae2-8a88-927d12a9e41b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:29.984830 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea6e4d-2bb3-4ae2-8a88-927d12a9e41b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:30.008507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1271/2511] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:12:29 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:12:30.013216 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097e1fb-bc8e-4a48-a655-55c105b52b4a admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:30.024487 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097e1fb-bc8e-4a48-a655-55c105b52b4a admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:30.029915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a7420-ebdd-4165-ade9-a8a43622c3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:30.030652 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a7420-ebdd-4165-ade9-a8a43622c3ba admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:30.030946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a7420-ebdd-4165-ade9-a8a43622c3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:30.031223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a7420-ebdd-4165-ade9-a8a43622c3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:30.031952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a7420-ebdd-4165-ade9-a8a43622c3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:30.033868 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a7420-ebdd-4165-ade9-a8a43622c3ba admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:30.048274 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1241/2512] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:12:30.049512 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea6e4d-2bb3-4ae2-8a88-927d12a9e41b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:30.059919 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea6e4d-2bb3-4ae2-8a88-927d12a9e41b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:30.065526 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebf42f-d057-47cc-927a-e50c8ccfbd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:30.066429 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebf42f-d057-47cc-927a-e50c8ccfbd4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:30.066707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebf42f-d057-47cc-927a-e50c8ccfbd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:30.066983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebf42f-d057-47cc-927a-e50c8ccfbd4f admin admin] PATH_INFO: `/v3/projects/0d4e2516f10740b1a845eb01069dded1/users/5d14f6a408e441b794ddce9479328ffa/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:30.084893 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebf42f-d057-47cc-927a-e50c8ccfbd4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d4e2516f10740b1a845eb01069dded1, user_id=5d14f6a408e441b794ddce9479328ffa, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:30.087550 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebf42f-d057-47cc-927a-e50c8ccfbd4f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:30.104769 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1272/2513] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:12:30 2022] PUT /identity/v3/projects/0d4e2516f10740b1a845eb01069dded1/users/5d14f6a408e441b794ddce9479328ffa/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:12:30.108617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a7420-ebdd-4165-ade9-a8a43622c3ba admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:30.119324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a7420-ebdd-4165-ade9-a8a43622c3ba admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:30.176981 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:30.192618 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0d70c-c3a9-4a5c-9f78-495aaaed906b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:30.193366 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0d70c-c3a9-4a5c-9f78-495aaaed906b admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:30.193819 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0d70c-c3a9-4a5c-9f78-495aaaed906b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:30.194121 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0d70c-c3a9-4a5c-9f78-495aaaed906b admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:30.194883 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0d70c-c3a9-4a5c-9f78-495aaaed906b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:30.196639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0d70c-c3a9-4a5c-9f78-495aaaed906b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:30.208055 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1242/2514] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:12:30.212267 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebf42f-d057-47cc-927a-e50c8ccfbd4f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:30.226282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebf42f-d057-47cc-927a-e50c8ccfbd4f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:30.237167 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102d559-2319-4e0a-a8c7-c2f616ebe134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:30.238088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102d559-2319-4e0a-a8c7-c2f616ebe134 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:30.238523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102d559-2319-4e0a-a8c7-c2f616ebe134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:30.238880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102d559-2319-4e0a-a8c7-c2f616ebe134 admin admin] PATH_INFO: `/v3/projects/0d4e2516f10740b1a845eb01069dded1/users/5d14f6a408e441b794ddce9479328ffa/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:30.241226 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102d559-2319-4e0a-a8c7-c2f616ebe134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d4e2516f10740b1a845eb01069dded1, user_id=5d14f6a408e441b794ddce9479328ffa, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:30.244359 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102d559-2319-4e0a-a8c7-c2f616ebe134 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:30.257513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1273/2515] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:12:30 2022] PUT /identity/v3/projects/0d4e2516f10740b1a845eb01069dded1/users/5d14f6a408e441b794ddce9479328ffa/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:12:30.270432 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cac361-29ad-4676-8808-abf81d0130e3 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:30.270781 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cac361-29ad-4676-8808-abf81d0130e3 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:30.271130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cac361-29ad-4676-8808-abf81d0130e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:30.303714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-82cac361-29ad-4676-8808-abf81d0130e3 None None] MFA Rules not processed for user `5d14f6a408e441b794ddce9479328ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:30.305011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-82cac361-29ad-4676-8808-abf81d0130e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:30.375990 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1243/2516] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:30 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:30.387240 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102d559-2319-4e0a-a8c7-c2f616ebe134 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:30.389577 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7102d559-2319-4e0a-a8c7-c2f616ebe134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:30.450456 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102d559-2319-4e0a-a8c7-c2f616ebe134 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:30.455278 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3400b44-cc64-4b35-959f-f38bf880c531 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:30.455958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3400b44-cc64-4b35-959f-f38bf880c531 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:30.456265 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3400b44-cc64-4b35-959f-f38bf880c531 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:30.456633 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3400b44-cc64-4b35-959f-f38bf880c531 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:30.466567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3400b44-cc64-4b35-959f-f38bf880c531 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:30.468239 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3400b44-cc64-4b35-959f-f38bf880c531 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:30.472193 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:30.479649 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1274/2517] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:12:30 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:12:31.947931 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cac361-29ad-4676-8808-abf81d0130e3 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:32.023959 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cac361-29ad-4676-8808-abf81d0130e3 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:32.029215 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bea323-93ae-451a-813b-660202695f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:32.030327 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bea323-93ae-451a-813b-660202695f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:32.030609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bea323-93ae-451a-813b-660202695f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:32.030886 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bea323-93ae-451a-813b-660202695f90 admin admin] PATH_INFO: `/v3/users/db7cd6475b31454293f142137b8e6fe1` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:32.032184 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bea323-93ae-451a-813b-660202695f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db7cd6475b31454293f142137b8e6fe1)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:32.034302 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bea323-93ae-451a-813b-660202695f90 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:32.087901 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-70bea323-93ae-451a-813b-660202695f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db7cd6475b31454293f142137b8e6fe1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:32.088683 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-70bea323-93ae-451a-813b-660202695f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db7cd6475b31454293f142137b8e6fe1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:32.098287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-70bea323-93ae-451a-813b-660202695f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db7cd6475b31454293f142137b8e6fe1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:32.108062 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-70bea323-93ae-451a-813b-660202695f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db7cd6475b31454293f142137b8e6fe1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:32.109000 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-70bea323-93ae-451a-813b-660202695f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db7cd6475b31454293f142137b8e6fe1'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:32.116637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:32.137003 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1244/2518] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:12:31 2022] DELETE /identity/v3/users/db7cd6475b31454293f142137b8e6fe1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:12:32.140256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679f7cc-c8cf-4cf7-8ad3-79875d106554 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:32.140809 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679f7cc-c8cf-4cf7-8ad3-79875d106554 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:32.140931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679f7cc-c8cf-4cf7-8ad3-79875d106554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:32.161812 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-4679f7cc-c8cf-4cf7-8ad3-79875d106554 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:32.163317 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4679f7cc-c8cf-4cf7-8ad3-79875d106554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:32.171449 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89b265-31f8-499d-8c8e-3413455d74b8 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:32.172011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89b265-31f8-499d-8c8e-3413455d74b8 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:32.172289 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89b265-31f8-499d-8c8e-3413455d74b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:32.194552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5f89b265-31f8-499d-8c8e-3413455d74b8 None None] MFA Rules not processed for user `28b3e73439e64050b406a9f3e9b0160a`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:32.195686 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5f89b265-31f8-499d-8c8e-3413455d74b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:32.249129 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1275/2519] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:32.259181 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679f7cc-c8cf-4cf7-8ad3-79875d106554 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:32.260803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4679f7cc-c8cf-4cf7-8ad3-79875d106554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:32.270534 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1245/2520] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Jun 25 16:12:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:12:32.332479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89b265-31f8-499d-8c8e-3413455d74b8 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:32.334104 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5f89b265-31f8-499d-8c8e-3413455d74b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:32.338336 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679f7cc-c8cf-4cf7-8ad3-79875d106554 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:32.342759 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf100c-b683-4342-b299-278b867883d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:32.343413 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf100c-b683-4342-b299-278b867883d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:32.343859 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf100c-b683-4342-b299-278b867883d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:32.344320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf100c-b683-4342-b299-278b867883d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:32.354646 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf100c-b683-4342-b299-278b867883d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:32.356212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf100c-b683-4342-b299-278b867883d7 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:32.367613 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1276/2521] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:12:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:12:32.432534 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89b265-31f8-499d-8c8e-3413455d74b8 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:32.438520 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47131851-ee9c-4224-83e4-bbf56e7d6567 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19788c4239ca44f9a31a7d32086c8947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:32.439587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47131851-ee9c-4224-83e4-bbf56e7d6567 service swift] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:32.440045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47131851-ee9c-4224-83e4-bbf56e7d6567 service swift] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:32.440653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47131851-ee9c-4224-83e4-bbf56e7d6567 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:32.453856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47131851-ee9c-4224-83e4-bbf56e7d6567 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:32.456655 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47131851-ee9c-4224-83e4-bbf56e7d6567 service swift] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:32.468113 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1246/2522] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Jun 25 16:12:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 25 16:12:32.726176 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf100c-b683-4342-b299-278b867883d7 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:32.727933 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-d5bf100c-b683-4342-b299-278b867883d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:32.748049 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf100c-b683-4342-b299-278b867883d7 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:32.761534 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f6df1-f2a7-4fa7-a1a2-07b9b325b523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:32.762931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f6df1-f2a7-4fa7-a1a2-07b9b325b523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:32.763219 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f6df1-f2a7-4fa7-a1a2-07b9b325b523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:32.763502 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f6df1-f2a7-4fa7-a1a2-07b9b325b523 admin admin] PATH_INFO: `/v3/projects/5c418ab33a154076b191667695ec955f` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:32.765673 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f6df1-f2a7-4fa7-a1a2-07b9b325b523 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c418ab33a154076b191667695ec955f)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:32.768976 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f6df1-f2a7-4fa7-a1a2-07b9b325b523 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:32.834421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-841f6df1-f2a7-4fa7-a1a2-07b9b325b523 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c418ab33a154076b191667695ec955f', 'request_id': 'req-841f6df1-f2a7-4fa7-a1a2-07b9b325b523'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:32.845416 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-841f6df1-f2a7-4fa7-a1a2-07b9b325b523 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c418ab33a154076b191667695ec955f', 'request_id': 'req-841f6df1-f2a7-4fa7-a1a2-07b9b325b523'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:32.852177 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-841f6df1-f2a7-4fa7-a1a2-07b9b325b523 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c418ab33a154076b191667695ec955f', 'request_id': 'req-841f6df1-f2a7-4fa7-a1a2-07b9b325b523'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:32.870246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-841f6df1-f2a7-4fa7-a1a2-07b9b325b523 admin admin] The token cache is being invalidate because project 5c418ab33a154076b191667695ec955f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:12:32.870536 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-841f6df1-f2a7-4fa7-a1a2-07b9b325b523 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:32.871226 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-841f6df1-f2a7-4fa7-a1a2-07b9b325b523 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:32.872828 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1277/2523] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:12:32 2022] DELETE /identity/v3/projects/5c418ab33a154076b191667695ec955f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:12:32.877261 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47131851-ee9c-4224-83e4-bbf56e7d6567 service swift] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:32.891089 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47131851-ee9c-4224-83e4-bbf56e7d6567 service swift] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:32.958456 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e5324-6abf-4b62-9eeb-3b219de7f176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:32.959449 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e5324-6abf-4b62-9eeb-3b219de7f176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:32.959798 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e5324-6abf-4b62-9eeb-3b219de7f176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:32.960162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e5324-6abf-4b62-9eeb-3b219de7f176 admin admin] PATH_INFO: `/v3/users/5d14f6a408e441b794ddce9479328ffa` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:32.961508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e5324-6abf-4b62-9eeb-3b219de7f176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d14f6a408e441b794ddce9479328ffa)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:32.963876 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e5324-6abf-4b62-9eeb-3b219de7f176 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:32.990729 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:33.024648 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-773e5324-6abf-4b62-9eeb-3b219de7f176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d14f6a408e441b794ddce9479328ffa'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:33.027961 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-773e5324-6abf-4b62-9eeb-3b219de7f176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d14f6a408e441b794ddce9479328ffa'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:33.039761 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-773e5324-6abf-4b62-9eeb-3b219de7f176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d14f6a408e441b794ddce9479328ffa'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:33.050784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-773e5324-6abf-4b62-9eeb-3b219de7f176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d14f6a408e441b794ddce9479328ffa'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:33.051531 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-773e5324-6abf-4b62-9eeb-3b219de7f176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d14f6a408e441b794ddce9479328ffa'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:33.072740 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1247/2524] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:12:32 2022] DELETE /identity/v3/users/5d14f6a408e441b794ddce9479328ffa => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:12:33.402083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f6df1-f2a7-4fa7-a1a2-07b9b325b523 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:33.403861 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-841f6df1-f2a7-4fa7-a1a2-07b9b325b523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:33.423923 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f6df1-f2a7-4fa7-a1a2-07b9b325b523 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:33.452661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:33.495505 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbed0d0-6251-4ffb-b795-67c912c67e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:33.496386 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbed0d0-6251-4ffb-b795-67c912c67e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:33.496709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbed0d0-6251-4ffb-b795-67c912c67e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:33.496980 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbed0d0-6251-4ffb-b795-67c912c67e2e admin admin] PATH_INFO: `/v3/projects/0d4e2516f10740b1a845eb01069dded1` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:33.498270 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbed0d0-6251-4ffb-b795-67c912c67e2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d4e2516f10740b1a845eb01069dded1)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:33.500391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbed0d0-6251-4ffb-b795-67c912c67e2e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:33.558132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-dfbed0d0-6251-4ffb-b795-67c912c67e2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d4e2516f10740b1a845eb01069dded1', 'request_id': 'req-dfbed0d0-6251-4ffb-b795-67c912c67e2e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:33.564149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-dfbed0d0-6251-4ffb-b795-67c912c67e2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d4e2516f10740b1a845eb01069dded1', 'request_id': 'req-dfbed0d0-6251-4ffb-b795-67c912c67e2e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:33.571764 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-dfbed0d0-6251-4ffb-b795-67c912c67e2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d4e2516f10740b1a845eb01069dded1', 'request_id': 'req-dfbed0d0-6251-4ffb-b795-67c912c67e2e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:33.594923 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-dfbed0d0-6251-4ffb-b795-67c912c67e2e admin admin] The token cache is being invalidate because project 0d4e2516f10740b1a845eb01069dded1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:12:33.595513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-dfbed0d0-6251-4ffb-b795-67c912c67e2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:33.596368 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-dfbed0d0-6251-4ffb-b795-67c912c67e2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:33.598443 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1278/2525] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:12:33 2022] DELETE /identity/v3/projects/0d4e2516f10740b1a845eb01069dded1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:12:33.609059 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97a4ed-d627-4ca5-9264-4e8773847dc7 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:33.609549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97a4ed-d627-4ca5-9264-4e8773847dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:33.609970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97a4ed-d627-4ca5-9264-4e8773847dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:33.636111 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-fa97a4ed-d627-4ca5-9264-4e8773847dc7 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:33.637400 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-fa97a4ed-d627-4ca5-9264-4e8773847dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:33.721152 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1248/2526] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:33.732379 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e05cd-92e2-4715-ab49-adee4cebc6c7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:33.732379 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e05cd-92e2-4715-ab49-adee4cebc6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:33.732379 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e05cd-92e2-4715-ab49-adee4cebc6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:33.762400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-db3e05cd-92e2-4715-ab49-adee4cebc6c7 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:33.763621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-db3e05cd-92e2-4715-ab49-adee4cebc6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:33.775318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1279/2527] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:33.780650 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97a4ed-d627-4ca5-9264-4e8773847dc7 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:33.798351 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97a4ed-d627-4ca5-9264-4e8773847dc7 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:33.806397 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195dc4f-ee55-4443-a3b3-3efaf6f86f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:33.807684 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195dc4f-ee55-4443-a3b3-3efaf6f86f73 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:33.808036 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195dc4f-ee55-4443-a3b3-3efaf6f86f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:33.808436 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195dc4f-ee55-4443-a3b3-3efaf6f86f73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:33.809254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195dc4f-ee55-4443-a3b3-3efaf6f86f73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:12:33.809632 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195dc4f-ee55-4443-a3b3-3efaf6f86f73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:33.812114 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195dc4f-ee55-4443-a3b3-3efaf6f86f73 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:33.828438 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1249/2528] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:12:33.830315 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e05cd-92e2-4715-ab49-adee4cebc6c7 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:33.841281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e05cd-92e2-4715-ab49-adee4cebc6c7 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:33.848859 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256483a-eaf1-4693-a179-96f09218ebd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:33.849538 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256483a-eaf1-4693-a179-96f09218ebd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:33.849816 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256483a-eaf1-4693-a179-96f09218ebd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:33.850105 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256483a-eaf1-4693-a179-96f09218ebd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:33.850779 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256483a-eaf1-4693-a179-96f09218ebd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:33.852703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256483a-eaf1-4693-a179-96f09218ebd2 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:33.886642 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1280/2529] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:12:33 2022] POST /identity/v3/projects => generated 361 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:12:33.891169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195dc4f-ee55-4443-a3b3-3efaf6f86f73 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:33.904397 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195dc4f-ee55-4443-a3b3-3efaf6f86f73 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:33.980321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f94ff2-23fb-48d3-8684-a8296da50069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:33.981182 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f94ff2-23fb-48d3-8684-a8296da50069 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:33.981544 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f94ff2-23fb-48d3-8684-a8296da50069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:33.981892 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f94ff2-23fb-48d3-8684-a8296da50069 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:33.982748 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f94ff2-23fb-48d3-8684-a8296da50069 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:33.987967 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f94ff2-23fb-48d3-8684-a8296da50069 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:34.019313 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1250/2530] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:12:33 2022] POST /identity/v3/users => generated 281 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:12:34.020633 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256483a-eaf1-4693-a179-96f09218ebd2 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:34.032507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256483a-eaf1-4693-a179-96f09218ebd2 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:34.043597 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8efd11-b07e-4255-81e0-586b148ea12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:34.044304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8efd11-b07e-4255-81e0-586b148ea12a admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:34.044734 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8efd11-b07e-4255-81e0-586b148ea12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:34.045245 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8efd11-b07e-4255-81e0-586b148ea12a admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:34.046797 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8efd11-b07e-4255-81e0-586b148ea12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:34.048495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8efd11-b07e-4255-81e0-586b148ea12a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:34.069057 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1281/2531] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:12:34.070985 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f94ff2-23fb-48d3-8684-a8296da50069 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:34.082711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f94ff2-23fb-48d3-8684-a8296da50069 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:34.087697 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec900c31-1569-48a3-8ef4-97255258c1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:34.088383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec900c31-1569-48a3-8ef4-97255258c1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:34.088722 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec900c31-1569-48a3-8ef4-97255258c1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:34.088998 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec900c31-1569-48a3-8ef4-97255258c1d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:34.089790 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec900c31-1569-48a3-8ef4-97255258c1d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:34.091449 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec900c31-1569-48a3-8ef4-97255258c1d7 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:34.104511 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1251/2532] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:12:34.113541 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8efd11-b07e-4255-81e0-586b148ea12a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:34.119878 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec900c31-1569-48a3-8ef4-97255258c1d7 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:34.127258 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8efd11-b07e-4255-81e0-586b148ea12a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:34.128936 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ec900c31-1569-48a3-8ef4-97255258c1d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:34.133808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8db88-99f5-43fb-9a93-298a0d7e3ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:34.134892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8db88-99f5-43fb-9a93-298a0d7e3ec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:34.135335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8db88-99f5-43fb-9a93-298a0d7e3ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:34.136910 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8db88-99f5-43fb-9a93-298a0d7e3ec0 admin admin] PATH_INFO: `/v3/projects/7ec709a839754db6b135298221de78a9/users/53e0fc2fba064de2bf703b1f820a66b5/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:34.141623 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:34.160733 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8db88-99f5-43fb-9a93-298a0d7e3ec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec709a839754db6b135298221de78a9, user_id=53e0fc2fba064de2bf703b1f820a66b5, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:34.166567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8db88-99f5-43fb-9a93-298a0d7e3ec0 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:34.194744 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1282/2533] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:12:34 2022] PUT /identity/v3/projects/7ec709a839754db6b135298221de78a9/users/53e0fc2fba064de2bf703b1f820a66b5/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:12:34.205640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c299c74-c7a3-43cf-9b4c-3697b329a5d3 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:34.205972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c299c74-c7a3-43cf-9b4c-3697b329a5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:34.206504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c299c74-c7a3-43cf-9b4c-3697b329a5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:34.226047 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec900c31-1569-48a3-8ef4-97255258c1d7 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:34.243948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-2c299c74-c7a3-43cf-9b4c-3697b329a5d3 None None] MFA Rules not processed for user `53e0fc2fba064de2bf703b1f820a66b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:34.245256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2c299c74-c7a3-43cf-9b4c-3697b329a5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:34.300761 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:34.321208 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c494f04-139c-4794-83cc-32e7f3db5018 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:34.322163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c494f04-139c-4794-83cc-32e7f3db5018 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:34.322587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c494f04-139c-4794-83cc-32e7f3db5018 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:34.322877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c494f04-139c-4794-83cc-32e7f3db5018 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:34.325072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4c494f04-139c-4794-83cc-32e7f3db5018 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:34.338528 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c494f04-139c-4794-83cc-32e7f3db5018 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:34.340276 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c494f04-139c-4794-83cc-32e7f3db5018 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:34.340727 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1283/2534] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:34.353193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1252/2535] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:12:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:12:34.388256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c299c74-c7a3-43cf-9b4c-3697b329a5d3 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:34.390377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2c299c74-c7a3-43cf-9b4c-3697b329a5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:34.456506 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c299c74-c7a3-43cf-9b4c-3697b329a5d3 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:34.460400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd92e36-8ac8-457d-ad81-783ff0a77d10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:34.461153 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd92e36-8ac8-457d-ad81-783ff0a77d10 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:34.461499 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd92e36-8ac8-457d-ad81-783ff0a77d10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:34.461835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd92e36-8ac8-457d-ad81-783ff0a77d10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:34.470837 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd92e36-8ac8-457d-ad81-783ff0a77d10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:34.472481 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd92e36-8ac8-457d-ad81-783ff0a77d10 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:34.482624 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1284/2536] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:12:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:12:34.497504 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c494f04-139c-4794-83cc-32e7f3db5018 service nova] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:34.569706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c494f04-139c-4794-83cc-32e7f3db5018 service nova] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:34.577424 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f5174-aebf-4062-9d49-08cd338a9af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:34.578350 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f5174-aebf-4062-9d49-08cd338a9af6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:34.578817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f5174-aebf-4062-9d49-08cd338a9af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:34.579243 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f5174-aebf-4062-9d49-08cd338a9af6 admin admin] PATH_INFO: `/v3/users/53e0fc2fba064de2bf703b1f820a66b5` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:34.580537 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f5174-aebf-4062-9d49-08cd338a9af6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53e0fc2fba064de2bf703b1f820a66b5)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:34.582462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f5174-aebf-4062-9d49-08cd338a9af6 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:34.626753 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:34.643774 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-871f5174-aebf-4062-9d49-08cd338a9af6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53e0fc2fba064de2bf703b1f820a66b5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:34.644723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-871f5174-aebf-4062-9d49-08cd338a9af6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53e0fc2fba064de2bf703b1f820a66b5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:34.654754 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-871f5174-aebf-4062-9d49-08cd338a9af6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53e0fc2fba064de2bf703b1f820a66b5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:34.663018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-871f5174-aebf-4062-9d49-08cd338a9af6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53e0fc2fba064de2bf703b1f820a66b5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:34.663813 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-871f5174-aebf-4062-9d49-08cd338a9af6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53e0fc2fba064de2bf703b1f820a66b5'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:34.697651 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1253/2537] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:12:34 2022] DELETE /identity/v3/users/53e0fc2fba064de2bf703b1f820a66b5 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:12:34.699483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16168824-240c-4c46-a255-1ff7437f6646 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:34.699880 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16168824-240c-4c46-a255-1ff7437f6646 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:34.700166 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16168824-240c-4c46-a255-1ff7437f6646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:34.729557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-16168824-240c-4c46-a255-1ff7437f6646 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:34.731262 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-16168824-240c-4c46-a255-1ff7437f6646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:34.807635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1285/2538] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:34.820096 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f5174-aebf-4062-9d49-08cd338a9af6 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:34.822073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-871f5174-aebf-4062-9d49-08cd338a9af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:34.933633 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f5174-aebf-4062-9d49-08cd338a9af6 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:34.942337 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a85f8d-18cc-4b5d-a1d9-ad0fc6f4250d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:34.943149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a85f8d-18cc-4b5d-a1d9-ad0fc6f4250d service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:34.943531 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a85f8d-18cc-4b5d-a1d9-ad0fc6f4250d service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:34.943928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a85f8d-18cc-4b5d-a1d9-ad0fc6f4250d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:34.959064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a85f8d-18cc-4b5d-a1d9-ad0fc6f4250d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:34.959064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a85f8d-18cc-4b5d-a1d9-ad0fc6f4250d service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:34.975494 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1254/2539] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:12:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:12:35.346314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16168824-240c-4c46-a255-1ff7437f6646 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:35.348068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-16168824-240c-4c46-a255-1ff7437f6646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:35.368742 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16168824-240c-4c46-a255-1ff7437f6646 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:35.377410 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b41b1a-2259-4f01-bb3b-7a6886187707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:35.380658 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b41b1a-2259-4f01-bb3b-7a6886187707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:35.381226 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b41b1a-2259-4f01-bb3b-7a6886187707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:35.381721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b41b1a-2259-4f01-bb3b-7a6886187707 admin admin] PATH_INFO: `/v3/projects/7ec709a839754db6b135298221de78a9` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:35.383259 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b41b1a-2259-4f01-bb3b-7a6886187707 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ec709a839754db6b135298221de78a9)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:35.385968 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b41b1a-2259-4f01-bb3b-7a6886187707 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:35.489138 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d8b41b1a-2259-4f01-bb3b-7a6886187707 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ec709a839754db6b135298221de78a9', 'request_id': 'req-d8b41b1a-2259-4f01-bb3b-7a6886187707'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:35.495932 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d8b41b1a-2259-4f01-bb3b-7a6886187707 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ec709a839754db6b135298221de78a9', 'request_id': 'req-d8b41b1a-2259-4f01-bb3b-7a6886187707'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:35.506335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d8b41b1a-2259-4f01-bb3b-7a6886187707 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ec709a839754db6b135298221de78a9', 'request_id': 'req-d8b41b1a-2259-4f01-bb3b-7a6886187707'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:35.523931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d8b41b1a-2259-4f01-bb3b-7a6886187707 admin admin] The token cache is being invalidate because project 7ec709a839754db6b135298221de78a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:12:35.524538 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d8b41b1a-2259-4f01-bb3b-7a6886187707 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:35.525662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-d8b41b1a-2259-4f01-bb3b-7a6886187707 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:12:35.527949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1286/2540] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:12:35 2022] DELETE /identity/v3/projects/7ec709a839754db6b135298221de78a9 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:12:35.535562 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbae842-4e17-47b5-a478-08b1cb04bd8b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:35.536309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbae842-4e17-47b5-a478-08b1cb04bd8b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:35.536309 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbae842-4e17-47b5-a478-08b1cb04bd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:35.559498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ffbae842-4e17-47b5-a478-08b1cb04bd8b None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:35.560828 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ffbae842-4e17-47b5-a478-08b1cb04bd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:35.688744 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1255/2541] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:12:35.696586 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5350e5-afb7-4433-9d41-85dbee3250c2 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:35.696870 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5350e5-afb7-4433-9d41-85dbee3250c2 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:35.697141 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5350e5-afb7-4433-9d41-85dbee3250c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:35.713312 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:35.731363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8d5350e5-afb7-4433-9d41-85dbee3250c2 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:35.733181 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8d5350e5-afb7-4433-9d41-85dbee3250c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:35.744094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1287/2542] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:35.749522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbae842-4e17-47b5-a478-08b1cb04bd8b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:35.760552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbae842-4e17-47b5-a478-08b1cb04bd8b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:35.766638 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2838b-0a30-4832-b17a-5f61bfb644a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:35.767343 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2838b-0a30-4832-b17a-5f61bfb644a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:35.767625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2838b-0a30-4832-b17a-5f61bfb644a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:35.767904 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2838b-0a30-4832-b17a-5f61bfb644a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:35.768688 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2838b-0a30-4832-b17a-5f61bfb644a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:12:35.768999 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2838b-0a30-4832-b17a-5f61bfb644a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:35.770629 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2838b-0a30-4832-b17a-5f61bfb644a1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:35.785100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1256/2543] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:12:35.789267 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5350e5-afb7-4433-9d41-85dbee3250c2 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:35.800004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5350e5-afb7-4433-9d41-85dbee3250c2 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:35.804769 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adc803-55cd-4d13-8e02-534680a11ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:35.805432 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adc803-55cd-4d13-8e02-534680a11ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:35.805703 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adc803-55cd-4d13-8e02-534680a11ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:35.805982 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adc803-55cd-4d13-8e02-534680a11ea8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:35.806683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adc803-55cd-4d13-8e02-534680a11ea8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:35.808602 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adc803-55cd-4d13-8e02-534680a11ea8 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:35.837565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1288/2544] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:12:35 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:12:35.845409 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2838b-0a30-4832-b17a-5f61bfb644a1 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:35.868054 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2838b-0a30-4832-b17a-5f61bfb644a1 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:35.917677 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:35.946982 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc68649-60da-4b54-890b-e082093fd272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:35.947879 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc68649-60da-4b54-890b-e082093fd272 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:35.948262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc68649-60da-4b54-890b-e082093fd272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:35.948717 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc68649-60da-4b54-890b-e082093fd272 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:35.949589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc68649-60da-4b54-890b-e082093fd272 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:35.952941 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc68649-60da-4b54-890b-e082093fd272 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:35.976666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1257/2545] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:12:35 2022] POST /identity/v3/users => generated 284 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:12:35.980474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adc803-55cd-4d13-8e02-534680a11ea8 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:35.994105 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adc803-55cd-4d13-8e02-534680a11ea8 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:36.005018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd60e80-b5ed-4f66-a3a3-993f777cfc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:36.005954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd60e80-b5ed-4f66-a3a3-993f777cfc45 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:36.006338 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd60e80-b5ed-4f66-a3a3-993f777cfc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:36.006667 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd60e80-b5ed-4f66-a3a3-993f777cfc45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:36.007534 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd60e80-b5ed-4f66-a3a3-993f777cfc45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:36.009190 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd60e80-b5ed-4f66-a3a3-993f777cfc45 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:36.027133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1289/2546] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:12:36.031053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc68649-60da-4b54-890b-e082093fd272 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:36.041524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc68649-60da-4b54-890b-e082093fd272 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:36.046920 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc4678-8645-4075-bb61-6f6d425f8add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:36.047586 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc4678-8645-4075-bb61-6f6d425f8add admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:36.047860 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc4678-8645-4075-bb61-6f6d425f8add admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:36.048133 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc4678-8645-4075-bb61-6f6d425f8add admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:36.048905 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc4678-8645-4075-bb61-6f6d425f8add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:36.050516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc4678-8645-4075-bb61-6f6d425f8add admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:36.061687 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1258/2547] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:12:36.066241 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd60e80-b5ed-4f66-a3a3-993f777cfc45 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:36.078761 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd60e80-b5ed-4f66-a3a3-993f777cfc45 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:36.086699 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09608e-3a4a-429c-8697-0969cd15d2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:36.087919 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09608e-3a4a-429c-8697-0969cd15d2a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:36.088089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09608e-3a4a-429c-8697-0969cd15d2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:36.088369 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09608e-3a4a-429c-8697-0969cd15d2a5 admin admin] PATH_INFO: `/v3/projects/a1a742509a164b798b894ad7240f28ed/users/a4e89444043b4a19ba07ab8b5617915a/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:36.110086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09608e-3a4a-429c-8697-0969cd15d2a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a742509a164b798b894ad7240f28ed, user_id=a4e89444043b4a19ba07ab8b5617915a, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:36.116538 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09608e-3a4a-429c-8697-0969cd15d2a5 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:36.133404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1290/2548] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:12:36 2022] PUT /identity/v3/projects/a1a742509a164b798b894ad7240f28ed/users/a4e89444043b4a19ba07ab8b5617915a/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:12:36.147109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee2d38-2c89-42f8-8e69-0e3a32439bc7 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:36.147430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee2d38-2c89-42f8-8e69-0e3a32439bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:36.147685 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee2d38-2c89-42f8-8e69-0e3a32439bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:36.178606 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2cee2d38-2c89-42f8-8e69-0e3a32439bc7 None None] MFA Rules not processed for user `a4e89444043b4a19ba07ab8b5617915a`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:36.179858 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2cee2d38-2c89-42f8-8e69-0e3a32439bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:36.255567 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1259/2549] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:12:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:36.269248 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09608e-3a4a-429c-8697-0969cd15d2a5 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:36.271108 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1d09608e-3a4a-429c-8697-0969cd15d2a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:36.356288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09608e-3a4a-429c-8697-0969cd15d2a5 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:36.361626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbd6d1-2ed0-4322-9a54-fe232bed4538 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cba086728824ec68fb0ee90fb3f99dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:36.362586 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbd6d1-2ed0-4322-9a54-fe232bed4538 service glance] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:36.362839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbd6d1-2ed0-4322-9a54-fe232bed4538 service glance] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:36.363145 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbd6d1-2ed0-4322-9a54-fe232bed4538 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:36.374955 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbd6d1-2ed0-4322-9a54-fe232bed4538 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:36.376567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbd6d1-2ed0-4322-9a54-fe232bed4538 service glance] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:36.392167 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1291/2550] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:12:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:12:44.543828 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee2d38-2c89-42f8-8e69-0e3a32439bc7 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:44.546422 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2cee2d38-2c89-42f8-8e69-0e3a32439bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:44.631478 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee2d38-2c89-42f8-8e69-0e3a32439bc7 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:44.639525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfda3c-eb99-492d-a4d5-4714199080b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:44.640453 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfda3c-eb99-492d-a4d5-4714199080b5 service nova] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:44.640810 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfda3c-eb99-492d-a4d5-4714199080b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:44.641190 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfda3c-eb99-492d-a4d5-4714199080b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:44.644184 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-70cfda3c-eb99-492d-a4d5-4714199080b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:44.698758 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:44.715804 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cfda3c-eb99-492d-a4d5-4714199080b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:44.717515 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cfda3c-eb99-492d-a4d5-4714199080b5 service nova] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:44.726811 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1260/2551] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:12:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:12:46.381720 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=50,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:47.135120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaceac-0699-4ced-b6a7-c63ca13665eb None None] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:47.135408 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaceac-0699-4ced-b6a7-c63ca13665eb None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:47.135707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaceac-0699-4ced-b6a7-c63ca13665eb None None] PATH_INFO: `/` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:47.137227 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1292/2552] 10.222.0.13 () {54 vars in 921 bytes} [Sat Jun 25 16:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 25 16:12:47.142639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5696d17-43f8-4c51-bbcb-0bf329da9f38 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:47.143125 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5696d17-43f8-4c51-bbcb-0bf329da9f38 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:47.143234 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5696d17-43f8-4c51-bbcb-0bf329da9f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:47.161428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-b5696d17-43f8-4c51-bbcb-0bf329da9f38 None None] MFA Rules not processed for user `3622c5ccb7534e7c8e8342b4f6643bc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:47.162711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b5696d17-43f8-4c51-bbcb-0bf329da9f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:47.172644 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1261/2553] 10.222.0.13 () {58 vars in 1018 bytes} [Sat Jun 25 16:12:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:47.184706 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaceac-0699-4ced-b6a7-c63ca13665eb None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:47.186565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b8aaceac-0699-4ced-b6a7-c63ca13665eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:47.202420 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaceac-0699-4ced-b6a7-c63ca13665eb None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:47.205614 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d05fc-6b4d-462a-bc2c-db112d2cf20e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cba086728824ec68fb0ee90fb3f99dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:47.206600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d05fc-6b4d-462a-bc2c-db112d2cf20e service glance] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:47.206878 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d05fc-6b4d-462a-bc2c-db112d2cf20e service glance] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:47.207149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d05fc-6b4d-462a-bc2c-db112d2cf20e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:47.220157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d05fc-6b4d-462a-bc2c-db112d2cf20e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:47.221751 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d05fc-6b4d-462a-bc2c-db112d2cf20e service glance] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:47.233557 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1293/2554] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:12:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:12:48.994855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105725a7-13f5-41a1-a99c-8dc126c86f83 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:48.995317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105725a7-13f5-41a1-a99c-8dc126c86f83 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:48.995406 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105725a7-13f5-41a1-a99c-8dc126c86f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:49.015484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-105725a7-13f5-41a1-a99c-8dc126c86f83 None None] MFA Rules not processed for user `28b3e73439e64050b406a9f3e9b0160a`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:12:49.016769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-105725a7-13f5-41a1-a99c-8dc126c86f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:49.075938 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1262/2555] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Jun 25 16:12:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:12:49.088299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d05fc-6b4d-462a-bc2c-db112d2cf20e service glance] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:12:49.089801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-de1d05fc-6b4d-462a-bc2c-db112d2cf20e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:12:49.170692 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d05fc-6b4d-462a-bc2c-db112d2cf20e service glance] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:12:49.177284 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fa0eb-073e-45bd-8b3f-0cd0e65742ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19788c4239ca44f9a31a7d32086c8947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:12:49.177930 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fa0eb-073e-45bd-8b3f-0cd0e65742ad service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:12:49.178209 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fa0eb-073e-45bd-8b3f-0cd0e65742ad service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:12:49.178483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fa0eb-073e-45bd-8b3f-0cd0e65742ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:12:49.187152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fa0eb-073e-45bd-8b3f-0cd0e65742ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:12:49.188776 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fa0eb-073e-45bd-8b3f-0cd0e65742ad service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:12:49.196736 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1294/2556] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Jun 25 16:12:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 25 16:12:59.068462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=35 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:12:59.193593 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=38 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:13:05.133402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6d9fb-7a67-40aa-aba7-2a5e7b87cdae None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:05.133912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6d9fb-7a67-40aa-aba7-2a5e7b87cdae None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:05.133912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6d9fb-7a67-40aa-aba7-2a5e7b87cdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:05.155229 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-aad6d9fb-7a67-40aa-aba7-2a5e7b87cdae None None] MFA Rules not processed for user `28b3e73439e64050b406a9f3e9b0160a`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:13:05.156473 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-aad6d9fb-7a67-40aa-aba7-2a5e7b87cdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:05.164550 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1263/2557] 10.222.0.13 () {58 vars in 967 bytes} [Sat Jun 25 16:13:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:13:05.179620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fa0eb-073e-45bd-8b3f-0cd0e65742ad service swift] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:05.192706 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fa0eb-073e-45bd-8b3f-0cd0e65742ad service swift] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:05.198276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d7320-1423-47d8-bbb4-b9a6135f29c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19788c4239ca44f9a31a7d32086c8947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:05.198958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d7320-1423-47d8-bbb4-b9a6135f29c4 service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:05.199272 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d7320-1423-47d8-bbb4-b9a6135f29c4 service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:05.199564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d7320-1423-47d8-bbb4-b9a6135f29c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:05.208524 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d7320-1423-47d8-bbb4-b9a6135f29c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:05.210110 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d7320-1423-47d8-bbb4-b9a6135f29c4 service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:05.218823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1295/2558] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Jun 25 16:13:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 25 16:13:08.194029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6d9fb-7a67-40aa-aba7-2a5e7b87cdae None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:08.195693 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-aad6d9fb-7a67-40aa-aba7-2a5e7b87cdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:08.258226 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6d9fb-7a67-40aa-aba7-2a5e7b87cdae None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:08.262455 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15729f5b-461e-49fa-8282-c51106393e5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:08.263408 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15729f5b-461e-49fa-8282-c51106393e5e service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:08.263917 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15729f5b-461e-49fa-8282-c51106393e5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:08.264083 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15729f5b-461e-49fa-8282-c51106393e5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:08.266418 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-15729f5b-461e-49fa-8282-c51106393e5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:08.327717 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15729f5b-461e-49fa-8282-c51106393e5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:08.329229 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15729f5b-461e-49fa-8282-c51106393e5e service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:08.339072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1264/2559] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:13:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:13:09.116981 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d7320-1423-47d8-bbb4-b9a6135f29c4 service swift] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:09.187754 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d7320-1423-47d8-bbb4-b9a6135f29c4 service swift] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:09.192863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a6698-5867-4cf0-9ed4-cefebe609901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:09.193867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a6698-5867-4cf0-9ed4-cefebe609901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:09.194052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a6698-5867-4cf0-9ed4-cefebe609901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:09.194302 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a6698-5867-4cf0-9ed4-cefebe609901 admin admin] PATH_INFO: `/v3/users/a4e89444043b4a19ba07ab8b5617915a` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:09.195701 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a6698-5867-4cf0-9ed4-cefebe609901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4e89444043b4a19ba07ab8b5617915a)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:09.199118 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a6698-5867-4cf0-9ed4-cefebe609901 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:09.259634 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3f4a6698-5867-4cf0-9ed4-cefebe609901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4e89444043b4a19ba07ab8b5617915a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:09.269271 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3f4a6698-5867-4cf0-9ed4-cefebe609901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4e89444043b4a19ba07ab8b5617915a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:09.270956 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3f4a6698-5867-4cf0-9ed4-cefebe609901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4e89444043b4a19ba07ab8b5617915a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:09.283206 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3f4a6698-5867-4cf0-9ed4-cefebe609901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4e89444043b4a19ba07ab8b5617915a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:09.283765 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3f4a6698-5867-4cf0-9ed4-cefebe609901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4e89444043b4a19ba07ab8b5617915a'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:09.325450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bf952-bc65-4741-9a46-5d205e9b8417 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:09.326490 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bf952-bc65-4741-9a46-5d205e9b8417 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:09.326769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bf952-bc65-4741-9a46-5d205e9b8417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:09.328614 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1296/2560] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:13:09 2022] DELETE /identity/v3/users/a4e89444043b4a19ba07ab8b5617915a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:13:09.345867 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-c49bf952-bc65-4741-9a46-5d205e9b8417 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:13:09.347070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c49bf952-bc65-4741-9a46-5d205e9b8417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:09.433271 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1265/2561] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:13:09.446899 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a6698-5867-4cf0-9ed4-cefebe609901 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:09.448623 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3f4a6698-5867-4cf0-9ed4-cefebe609901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:09.523684 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a6698-5867-4cf0-9ed4-cefebe609901 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:09.527351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821234b-6b62-4d82-8194-33a00b6b9df9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:09.528006 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821234b-6b62-4d82-8194-33a00b6b9df9 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:09.528285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821234b-6b62-4d82-8194-33a00b6b9df9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:09.528668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821234b-6b62-4d82-8194-33a00b6b9df9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:09.538504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821234b-6b62-4d82-8194-33a00b6b9df9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:09.539936 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821234b-6b62-4d82-8194-33a00b6b9df9 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:09.550283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1297/2562] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:13:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:13:09.832580 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bf952-bc65-4741-9a46-5d205e9b8417 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:09.834173 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c49bf952-bc65-4741-9a46-5d205e9b8417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:09.849702 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bf952-bc65-4741-9a46-5d205e9b8417 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:09.854893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073c06e-0385-40c7-8108-973e57ddcd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:09.855798 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073c06e-0385-40c7-8108-973e57ddcd6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:09.856209 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073c06e-0385-40c7-8108-973e57ddcd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:09.856509 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073c06e-0385-40c7-8108-973e57ddcd6a admin admin] PATH_INFO: `/v3/projects/a1a742509a164b798b894ad7240f28ed` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:09.858000 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073c06e-0385-40c7-8108-973e57ddcd6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a742509a164b798b894ad7240f28ed)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:09.860074 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073c06e-0385-40c7-8108-973e57ddcd6a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:09.914326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:13:09.918615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5073c06e-0385-40c7-8108-973e57ddcd6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a742509a164b798b894ad7240f28ed', 'request_id': 'req-5073c06e-0385-40c7-8108-973e57ddcd6a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:09.930904 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5073c06e-0385-40c7-8108-973e57ddcd6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a742509a164b798b894ad7240f28ed', 'request_id': 'req-5073c06e-0385-40c7-8108-973e57ddcd6a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:09.940289 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5073c06e-0385-40c7-8108-973e57ddcd6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a742509a164b798b894ad7240f28ed', 'request_id': 'req-5073c06e-0385-40c7-8108-973e57ddcd6a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:09.955668 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5073c06e-0385-40c7-8108-973e57ddcd6a admin admin] The token cache is being invalidate because project a1a742509a164b798b894ad7240f28ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:13:09.956044 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5073c06e-0385-40c7-8108-973e57ddcd6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:09.956728 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-5073c06e-0385-40c7-8108-973e57ddcd6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:09.958620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1266/2563] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:13:09 2022] DELETE /identity/v3/projects/a1a742509a164b798b894ad7240f28ed => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:13:09.963376 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d8cd4-ad60-45c8-a4f4-f6726d15a8b3 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:09.963867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d8cd4-ad60-45c8-a4f4-f6726d15a8b3 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:09.964290 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d8cd4-ad60-45c8-a4f4-f6726d15a8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:09.984689 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7c0d8cd4-ad60-45c8-a4f4-f6726d15a8b3 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:13:09.986118 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7c0d8cd4-ad60-45c8-a4f4-f6726d15a8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:09.991448 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:13:10.061170 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1298/2564] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:13:10.069193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae39c1-b840-479f-8870-d84eeff04cc9 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:10.069484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae39c1-b840-479f-8870-d84eeff04cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:10.069766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae39c1-b840-479f-8870-d84eeff04cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:10.089928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-24ae39c1-b840-479f-8870-d84eeff04cc9 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:13:10.091307 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-24ae39c1-b840-479f-8870-d84eeff04cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:10.103441 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1267/2565] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:13:10.104486 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d8cd4-ad60-45c8-a4f4-f6726d15a8b3 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:10.115150 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d8cd4-ad60-45c8-a4f4-f6726d15a8b3 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:10.120071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b324685-bb03-4597-ba10-545d1eb93cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:10.120777 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b324685-bb03-4597-ba10-545d1eb93cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:10.121062 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b324685-bb03-4597-ba10-545d1eb93cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:10.121340 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b324685-bb03-4597-ba10-545d1eb93cf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:10.122085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b324685-bb03-4597-ba10-545d1eb93cf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:13:10.122511 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b324685-bb03-4597-ba10-545d1eb93cf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:10.124865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b324685-bb03-4597-ba10-545d1eb93cf7 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:10.136883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1299/2566] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:13:10.141613 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae39c1-b840-479f-8870-d84eeff04cc9 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:10.151243 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae39c1-b840-479f-8870-d84eeff04cc9 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:10.156503 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10079be-848a-44e9-9403-f88a0ade2aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:10.157140 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10079be-848a-44e9-9403-f88a0ade2aca admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:10.157418 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10079be-848a-44e9-9403-f88a0ade2aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:10.157692 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10079be-848a-44e9-9403-f88a0ade2aca admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:10.158574 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10079be-848a-44e9-9403-f88a0ade2aca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:10.160304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10079be-848a-44e9-9403-f88a0ade2aca admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:10.189082 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1268/2567] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:13:10 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:13:10.193255 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b324685-bb03-4597-ba10-545d1eb93cf7 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:10.204662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b324685-bb03-4597-ba10-545d1eb93cf7 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:10.268803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a05c9-89e1-4a77-8f78-45ce37f004e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:10.269509 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a05c9-89e1-4a77-8f78-45ce37f004e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:10.269785 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a05c9-89e1-4a77-8f78-45ce37f004e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:10.270063 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a05c9-89e1-4a77-8f78-45ce37f004e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:10.270772 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a05c9-89e1-4a77-8f78-45ce37f004e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:10.272807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a05c9-89e1-4a77-8f78-45ce37f004e2 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:10.297034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1300/2568] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:13:10 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:13:10.298261 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10079be-848a-44e9-9403-f88a0ade2aca admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:10.309675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10079be-848a-44e9-9403-f88a0ade2aca admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:10.315492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67584da0-eaaf-4f83-a834-ce115db50f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:10.316147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67584da0-eaaf-4f83-a834-ce115db50f0b admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:10.316467 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67584da0-eaaf-4f83-a834-ce115db50f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:10.316737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67584da0-eaaf-4f83-a834-ce115db50f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:10.317471 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67584da0-eaaf-4f83-a834-ce115db50f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:10.319093 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67584da0-eaaf-4f83-a834-ce115db50f0b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:10.330477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1269/2569] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:13:10.334347 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a05c9-89e1-4a77-8f78-45ce37f004e2 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:10.346398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a05c9-89e1-4a77-8f78-45ce37f004e2 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:10.351455 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b78684-078a-40ae-bc03-334ac819e177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:10.352142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b78684-078a-40ae-bc03-334ac819e177 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:10.352442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b78684-078a-40ae-bc03-334ac819e177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:10.352725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b78684-078a-40ae-bc03-334ac819e177 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:10.353461 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b78684-078a-40ae-bc03-334ac819e177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:10.355101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b78684-078a-40ae-bc03-334ac819e177 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:10.366419 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1301/2570] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:13:10.370670 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67584da0-eaaf-4f83-a834-ce115db50f0b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:10.381515 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67584da0-eaaf-4f83-a834-ce115db50f0b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:10.386451 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640c8b0-6109-4df9-b8c7-8c779fe59f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:10.387326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640c8b0-6109-4df9-b8c7-8c779fe59f38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:10.387611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640c8b0-6109-4df9-b8c7-8c779fe59f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:10.387886 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640c8b0-6109-4df9-b8c7-8c779fe59f38 admin admin] PATH_INFO: `/v3/projects/c5bb30b042f3422faf43af82b27b37a5/users/0ef599055aed42f3a97fb30bba553fb0/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:10.403936 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640c8b0-6109-4df9-b8c7-8c779fe59f38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5bb30b042f3422faf43af82b27b37a5, user_id=0ef599055aed42f3a97fb30bba553fb0, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:10.406995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7640c8b0-6109-4df9-b8c7-8c779fe59f38 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:10.422289 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1270/2571] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:13:10 2022] PUT /identity/v3/projects/c5bb30b042f3422faf43af82b27b37a5/users/0ef599055aed42f3a97fb30bba553fb0/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:13:10.427393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8fcfd-a10f-4102-85b0-5f7d9e6abef5 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:10.427665 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8fcfd-a10f-4102-85b0-5f7d9e6abef5 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:10.427942 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d8fcfd-a10f-4102-85b0-5f7d9e6abef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:10.448198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-83d8fcfd-a10f-4102-85b0-5f7d9e6abef5 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:13:10.449468 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-83d8fcfd-a10f-4102-85b0-5f7d9e6abef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:10.513992 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1302/2572] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:13:10.524707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640c8b0-6109-4df9-b8c7-8c779fe59f38 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:10.526480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7640c8b0-6109-4df9-b8c7-8c779fe59f38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:10.588766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640c8b0-6109-4df9-b8c7-8c779fe59f38 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:10.592845 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8b72b6-ca22-481b-a81a-c8b4fb353568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:10.593630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b72b6-ca22-481b-a81a-c8b4fb353568 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:10.593906 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b72b6-ca22-481b-a81a-c8b4fb353568 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:10.594193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b72b6-ca22-481b-a81a-c8b4fb353568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:10.604981 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8b72b6-ca22-481b-a81a-c8b4fb353568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:10.606461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8b72b6-ca22-481b-a81a-c8b4fb353568 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:10.616244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1271/2573] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:13:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:13:13.395653 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8fcfd-a10f-4102-85b0-5f7d9e6abef5 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:13.397176 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-83d8fcfd-a10f-4102-85b0-5f7d9e6abef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:13.415568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d8fcfd-a10f-4102-85b0-5f7d9e6abef5 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:13.428506 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8079b5e-d4d2-46a0-9b01-b89d2ab977de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:13.429535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8079b5e-d4d2-46a0-9b01-b89d2ab977de service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:13.430009 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8079b5e-d4d2-46a0-9b01-b89d2ab977de service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:13.430379 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8079b5e-d4d2-46a0-9b01-b89d2ab977de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:13.438149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b8079b5e-d4d2-46a0-9b01-b89d2ab977de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:13.468123 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8079b5e-d4d2-46a0-9b01-b89d2ab977de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:13.469957 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8079b5e-d4d2-46a0-9b01-b89d2ab977de service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:13.489352 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1303/2574] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:13:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:13:15.182578 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3881331-b7ac-4018-bd89-082fad2641cc None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:15.183033 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3881331-b7ac-4018-bd89-082fad2641cc None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:15.183033 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3881331-b7ac-4018-bd89-082fad2641cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:15.212847 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d3881331-b7ac-4018-bd89-082fad2641cc None None] MFA Rules not processed for user `0ef599055aed42f3a97fb30bba553fb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:13:15.214002 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d3881331-b7ac-4018-bd89-082fad2641cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:15.300958 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:13:15.309559 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8079b5e-d4d2-46a0-9b01-b89d2ab977de service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:15.314904 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:13:15.315395 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1272/2575] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:13:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:13:15.324228 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8079b5e-d4d2-46a0-9b01-b89d2ab977de service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:15.331416 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab09049-09d2-49a2-9644-3862b6e7a710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:15.332471 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab09049-09d2-49a2-9644-3862b6e7a710 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:15.332917 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab09049-09d2-49a2-9644-3862b6e7a710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:15.333358 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab09049-09d2-49a2-9644-3862b6e7a710 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:15.334202 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab09049-09d2-49a2-9644-3862b6e7a710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:15.336434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab09049-09d2-49a2-9644-3862b6e7a710 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:15.374286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1304/2576] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:13:15 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:13:15.378238 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3881331-b7ac-4018-bd89-082fad2641cc None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:15.397470 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3881331-b7ac-4018-bd89-082fad2641cc None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:15.485191 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7337e7-5375-46ea-9494-635feddc5e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:15.485191 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7337e7-5375-46ea-9494-635feddc5e15 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:15.485191 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7337e7-5375-46ea-9494-635feddc5e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:15.485694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7337e7-5375-46ea-9494-635feddc5e15 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:15.486499 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7337e7-5375-46ea-9494-635feddc5e15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:15.488611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7337e7-5375-46ea-9494-635feddc5e15 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:15.519307 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1273/2577] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:13:15 2022] POST /identity/v3/users => generated 296 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:13:15.520467 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab09049-09d2-49a2-9644-3862b6e7a710 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:15.531608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab09049-09d2-49a2-9644-3862b6e7a710 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:15.538526 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03d701-4957-44d3-b986-816c3aea6f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:15.539506 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03d701-4957-44d3-b986-816c3aea6f04 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:15.540109 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03d701-4957-44d3-b986-816c3aea6f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:15.541027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03d701-4957-44d3-b986-816c3aea6f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:15.542271 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03d701-4957-44d3-b986-816c3aea6f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:15.545136 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03d701-4957-44d3-b986-816c3aea6f04 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:15.563227 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1305/2578] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:13:15.564557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7337e7-5375-46ea-9494-635feddc5e15 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:15.576599 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7337e7-5375-46ea-9494-635feddc5e15 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:15.582215 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c94872-cd0e-40f3-add0-0f76b3b93bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:15.583348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c94872-cd0e-40f3-add0-0f76b3b93bcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:15.583787 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c94872-cd0e-40f3-add0-0f76b3b93bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:15.584224 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c94872-cd0e-40f3-add0-0f76b3b93bcb admin admin] PATH_INFO: `/v3/projects/cabbc5fd45484def9cc8f536d576d517/users/621480ae82d840a3927d879cbc375d7c/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:15.604045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c94872-cd0e-40f3-add0-0f76b3b93bcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cabbc5fd45484def9cc8f536d576d517, user_id=621480ae82d840a3927d879cbc375d7c, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:15.607692 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c94872-cd0e-40f3-add0-0f76b3b93bcb admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:15.630005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1274/2579] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:13:15 2022] PUT /identity/v3/projects/cabbc5fd45484def9cc8f536d576d517/users/621480ae82d840a3927d879cbc375d7c/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:13:15.631307 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03d701-4957-44d3-b986-816c3aea6f04 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:15.644900 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03d701-4957-44d3-b986-816c3aea6f04 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:15.729210 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d7c96-4bf2-4222-a0b1-854d30449c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:15.729897 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d7c96-4bf2-4222-a0b1-854d30449c09 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:15.730178 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d7c96-4bf2-4222-a0b1-854d30449c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:15.730455 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d7c96-4bf2-4222-a0b1-854d30449c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:15.731187 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d7c96-4bf2-4222-a0b1-854d30449c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:15.735799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d7c96-4bf2-4222-a0b1-854d30449c09 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:15.756899 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1306/2580] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:13:15.758323 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c94872-cd0e-40f3-add0-0f76b3b93bcb admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:15.770071 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c94872-cd0e-40f3-add0-0f76b3b93bcb admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:15.777120 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e1123-3569-4d71-bdbf-f694dddc430c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:15.778605 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e1123-3569-4d71-bdbf-f694dddc430c admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:15.779080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e1123-3569-4d71-bdbf-f694dddc430c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:15.779589 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e1123-3569-4d71-bdbf-f694dddc430c admin admin] PATH_INFO: `/v3/projects/cabbc5fd45484def9cc8f536d576d517/users/621480ae82d840a3927d879cbc375d7c/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:15.791137 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e1123-3569-4d71-bdbf-f694dddc430c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cabbc5fd45484def9cc8f536d576d517, user_id=621480ae82d840a3927d879cbc375d7c, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:15.795710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e1123-3569-4d71-bdbf-f694dddc430c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:15.808311 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1275/2581] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:13:15 2022] PUT /identity/v3/projects/cabbc5fd45484def9cc8f536d576d517/users/621480ae82d840a3927d879cbc375d7c/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:13:17.656685 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d7c96-4bf2-4222-a0b1-854d30449c09 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:17.658655 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-040d7c96-4bf2-4222-a0b1-854d30449c09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:17.831337 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d7c96-4bf2-4222-a0b1-854d30449c09 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:17.983626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f727f1c-70b3-40b7-b441-2223aa69ad14 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474c468a610140f8a5b18d5a1ed1d18e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:17.985389 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f727f1c-70b3-40b7-b441-2223aa69ad14 service placement] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:17.985913 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f727f1c-70b3-40b7-b441-2223aa69ad14 service placement] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:17.987109 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f727f1c-70b3-40b7-b441-2223aa69ad14 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:17.989429 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4f727f1c-70b3-40b7-b441-2223aa69ad14 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:18.011778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f727f1c-70b3-40b7-b441-2223aa69ad14 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:18.014595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f727f1c-70b3-40b7-b441-2223aa69ad14 service placement] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:18.020311 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:13:18.042090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1307/2582] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:13:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:13:21.887658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee953d5e-fa85-4f09-833f-80464b3768ca None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:21.888931 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee953d5e-fa85-4f09-833f-80464b3768ca None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:21.890773 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee953d5e-fa85-4f09-833f-80464b3768ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:21.934987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-ee953d5e-fa85-4f09-833f-80464b3768ca None None] MFA Rules not processed for user `621480ae82d840a3927d879cbc375d7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:13:21.936560 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ee953d5e-fa85-4f09-833f-80464b3768ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:22.075664 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1276/2583] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:13:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:13:22.097290 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f727f1c-70b3-40b7-b441-2223aa69ad14 service placement] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:22.100530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4f727f1c-70b3-40b7-b441-2223aa69ad14 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:22.121258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee953d5e-fa85-4f09-833f-80464b3768ca None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:22.123458 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ee953d5e-fa85-4f09-833f-80464b3768ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:22.161500 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee953d5e-fa85-4f09-833f-80464b3768ca None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:22.177153 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29c455-712d-4378-b647-e0a1a40e116b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474c468a610140f8a5b18d5a1ed1d18e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:22.179334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29c455-712d-4378-b647-e0a1a40e116b service placement] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:22.179794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29c455-712d-4378-b647-e0a1a40e116b service placement] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:22.180072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29c455-712d-4378-b647-e0a1a40e116b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:22.185872 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3c29c455-712d-4378-b647-e0a1a40e116b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:22.203357 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29c455-712d-4378-b647-e0a1a40e116b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:22.207938 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29c455-712d-4378-b647-e0a1a40e116b service placement] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:22.220450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1277/2584] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:13:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:13:22.220450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29c455-712d-4378-b647-e0a1a40e116b service placement] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:22.221222 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3c29c455-712d-4378-b647-e0a1a40e116b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:22.260644 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:13:22.275949 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f727f1c-70b3-40b7-b441-2223aa69ad14 service placement] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:22.282614 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3214f6-e822-4af1-a445-4bd9fa645755 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:22.283285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3214f6-e822-4af1-a445-4bd9fa645755 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:22.283570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3214f6-e822-4af1-a445-4bd9fa645755 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:22.283837 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3214f6-e822-4af1-a445-4bd9fa645755 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:22.328967 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29c455-712d-4378-b647-e0a1a40e116b service placement] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:22.339397 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a975b1-ab69-4545-a706-5c1469d9f763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:22.340296 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a975b1-ab69-4545-a706-5c1469d9f763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:22.340602 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a975b1-ab69-4545-a706-5c1469d9f763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:22.340929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a975b1-ab69-4545-a706-5c1469d9f763 admin admin] PATH_INFO: `/v3/users/5bd682806c864805a3b410411839ca99` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:22.361657 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a975b1-ab69-4545-a706-5c1469d9f763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bd682806c864805a3b410411839ca99)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:22.363642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a975b1-ab69-4545-a706-5c1469d9f763 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:22.372361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3214f6-e822-4af1-a445-4bd9fa645755 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:22.374603 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3214f6-e822-4af1-a445-4bd9fa645755 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:22.385628 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1308/2585] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:13:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:13:22.435537 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d1a975b1-ab69-4545-a706-5c1469d9f763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bd682806c864805a3b410411839ca99'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:22.436244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d1a975b1-ab69-4545-a706-5c1469d9f763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bd682806c864805a3b410411839ca99'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:22.448484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d1a975b1-ab69-4545-a706-5c1469d9f763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bd682806c864805a3b410411839ca99'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:22.465107 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d1a975b1-ab69-4545-a706-5c1469d9f763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bd682806c864805a3b410411839ca99'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:22.465928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d1a975b1-ab69-4545-a706-5c1469d9f763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bd682806c864805a3b410411839ca99'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:22.490011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1278/2586] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:13:22 2022] DELETE /identity/v3/users/5bd682806c864805a3b410411839ca99 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:13:22.749264 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3214f6-e822-4af1-a445-4bd9fa645755 service nova] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:22.767517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3214f6-e822-4af1-a445-4bd9fa645755 service nova] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:22.847975 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8680355-e13c-4b31-8078-e00561d81e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:22.848918 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8680355-e13c-4b31-8078-e00561d81e2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:22.849199 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8680355-e13c-4b31-8078-e00561d81e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:22.849482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8680355-e13c-4b31-8078-e00561d81e2d admin admin] PATH_INFO: `/v3/projects/9b1a552fc6cc43bb80f8a5f951bfcf80` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:22.850738 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8680355-e13c-4b31-8078-e00561d81e2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b1a552fc6cc43bb80f8a5f951bfcf80)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:22.852725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8680355-e13c-4b31-8078-e00561d81e2d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:22.925299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:13:22.942287 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c8680355-e13c-4b31-8078-e00561d81e2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b1a552fc6cc43bb80f8a5f951bfcf80', 'request_id': 'req-c8680355-e13c-4b31-8078-e00561d81e2d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:22.952194 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c8680355-e13c-4b31-8078-e00561d81e2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b1a552fc6cc43bb80f8a5f951bfcf80', 'request_id': 'req-c8680355-e13c-4b31-8078-e00561d81e2d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:22.959558 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c8680355-e13c-4b31-8078-e00561d81e2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b1a552fc6cc43bb80f8a5f951bfcf80', 'request_id': 'req-c8680355-e13c-4b31-8078-e00561d81e2d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:22.983270 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c8680355-e13c-4b31-8078-e00561d81e2d admin admin] The token cache is being invalidate because project 9b1a552fc6cc43bb80f8a5f951bfcf80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:13:22.983617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c8680355-e13c-4b31-8078-e00561d81e2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:22.984598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c8680355-e13c-4b31-8078-e00561d81e2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:22.986502 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1309/2587] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:13:22 2022] DELETE /identity/v3/projects/9b1a552fc6cc43bb80f8a5f951bfcf80 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:13:22.990846 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a975b1-ab69-4545-a706-5c1469d9f763 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:22.992515 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d1a975b1-ab69-4545-a706-5c1469d9f763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:23.006623 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a975b1-ab69-4545-a706-5c1469d9f763 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:23.082924 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efda610-e6d2-46af-a486-fe1a1dacce54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:23.083827 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efda610-e6d2-46af-a486-fe1a1dacce54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:23.084090 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efda610-e6d2-46af-a486-fe1a1dacce54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:23.084362 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efda610-e6d2-46af-a486-fe1a1dacce54 admin admin] PATH_INFO: `/v3/users/5ac532f064934130afc11fff87c73617` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:23.085620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efda610-e6d2-46af-a486-fe1a1dacce54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ac532f064934130afc11fff87c73617)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:23.087706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efda610-e6d2-46af-a486-fe1a1dacce54 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:23.146042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:13:23.148946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9efda610-e6d2-46af-a486-fe1a1dacce54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ac532f064934130afc11fff87c73617'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:23.150005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9efda610-e6d2-46af-a486-fe1a1dacce54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ac532f064934130afc11fff87c73617'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:23.164667 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9efda610-e6d2-46af-a486-fe1a1dacce54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ac532f064934130afc11fff87c73617'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:23.180197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9efda610-e6d2-46af-a486-fe1a1dacce54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ac532f064934130afc11fff87c73617'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:23.181391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9efda610-e6d2-46af-a486-fe1a1dacce54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ac532f064934130afc11fff87c73617'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:23.208502 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1279/2588] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:13:22 2022] DELETE /identity/v3/users/5ac532f064934130afc11fff87c73617 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:13:23.423640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8680355-e13c-4b31-8078-e00561d81e2d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:23.425371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c8680355-e13c-4b31-8078-e00561d81e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:23.438883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8680355-e13c-4b31-8078-e00561d81e2d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:23.521955 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc506e-179f-4f61-b103-178b4fe74594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:23.522786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc506e-179f-4f61-b103-178b4fe74594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:23.523126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc506e-179f-4f61-b103-178b4fe74594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:23.523410 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddc506e-179f-4f61-b103-178b4fe74594 admin admin] PATH_INFO: `/v3/projects/45fcc90f2d094bffa9fc393df4548413` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:23.524680 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc506e-179f-4f61-b103-178b4fe74594 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45fcc90f2d094bffa9fc393df4548413)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:23.526773 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddc506e-179f-4f61-b103-178b4fe74594 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:23.588840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6ddc506e-179f-4f61-b103-178b4fe74594 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45fcc90f2d094bffa9fc393df4548413', 'request_id': 'req-6ddc506e-179f-4f61-b103-178b4fe74594'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:23.595167 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6ddc506e-179f-4f61-b103-178b4fe74594 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45fcc90f2d094bffa9fc393df4548413', 'request_id': 'req-6ddc506e-179f-4f61-b103-178b4fe74594'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:23.604757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6ddc506e-179f-4f61-b103-178b4fe74594 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45fcc90f2d094bffa9fc393df4548413', 'request_id': 'req-6ddc506e-179f-4f61-b103-178b4fe74594'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:23.624505 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6ddc506e-179f-4f61-b103-178b4fe74594 admin admin] The token cache is being invalidate because project 45fcc90f2d094bffa9fc393df4548413 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:13:23.625083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6ddc506e-179f-4f61-b103-178b4fe74594 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:23.626086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-6ddc506e-179f-4f61-b103-178b4fe74594 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:23.628864 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1310/2589] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:13:23 2022] DELETE /identity/v3/projects/45fcc90f2d094bffa9fc393df4548413 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:13:33.202466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=13,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:13:33.619242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:13:38.913316 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230cd45-07b2-41f9-8a97-0d1dc34acf22 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:38.913904 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230cd45-07b2-41f9-8a97-0d1dc34acf22 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:38.913904 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230cd45-07b2-41f9-8a97-0d1dc34acf22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:38.933992 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-2230cd45-07b2-41f9-8a97-0d1dc34acf22 None None] MFA Rules not processed for user `0ef599055aed42f3a97fb30bba553fb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:13:38.935242 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2230cd45-07b2-41f9-8a97-0d1dc34acf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:38.993969 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1280/2590] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:13:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:13:48.986544 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=25 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:13:51.430736 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc506e-179f-4f61-b103-178b4fe74594 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:51.432260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6ddc506e-179f-4f61-b103-178b4fe74594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:51.490832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddc506e-179f-4f61-b103-178b4fe74594 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:51.495145 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac299c-7a01-4b38-a2ac-5c9fd9c1d57b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:51.495806 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac299c-7a01-4b38-a2ac-5c9fd9c1d57b service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:51.496157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac299c-7a01-4b38-a2ac-5c9fd9c1d57b service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:51.496525 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac299c-7a01-4b38-a2ac-5c9fd9c1d57b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:51.498826 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-aaac299c-7a01-4b38-a2ac-5c9fd9c1d57b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:51.581792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac299c-7a01-4b38-a2ac-5c9fd9c1d57b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:51.583235 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac299c-7a01-4b38-a2ac-5c9fd9c1d57b service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:51.594214 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1311/2591] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:13:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:13:54.914572 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230cd45-07b2-41f9-8a97-0d1dc34acf22 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:54.916114 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-2230cd45-07b2-41f9-8a97-0d1dc34acf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:54.930630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230cd45-07b2-41f9-8a97-0d1dc34acf22 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:54.935528 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270ab8a-9c8b-4f62-af0b-a843c3df1c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:54.936473 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270ab8a-9c8b-4f62-af0b-a843c3df1c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:54.936713 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270ab8a-9c8b-4f62-af0b-a843c3df1c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:54.936987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270ab8a-9c8b-4f62-af0b-a843c3df1c58 admin admin] PATH_INFO: `/v3/users/744eb9d4e7084b7e9cae36d5de67f0f0` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:54.938308 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270ab8a-9c8b-4f62-af0b-a843c3df1c58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=744eb9d4e7084b7e9cae36d5de67f0f0)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:54.940252 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270ab8a-9c8b-4f62-af0b-a843c3df1c58 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:54.988565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b270ab8a-9c8b-4f62-af0b-a843c3df1c58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '744eb9d4e7084b7e9cae36d5de67f0f0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:54.989224 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b270ab8a-9c8b-4f62-af0b-a843c3df1c58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '744eb9d4e7084b7e9cae36d5de67f0f0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:54.998169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b270ab8a-9c8b-4f62-af0b-a843c3df1c58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '744eb9d4e7084b7e9cae36d5de67f0f0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:55.006338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b270ab8a-9c8b-4f62-af0b-a843c3df1c58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '744eb9d4e7084b7e9cae36d5de67f0f0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:55.007075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b270ab8a-9c8b-4f62-af0b-a843c3df1c58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '744eb9d4e7084b7e9cae36d5de67f0f0'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:55.025627 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1281/2592] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:13:54 2022] DELETE /identity/v3/users/744eb9d4e7084b7e9cae36d5de67f0f0 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:13:55.204610 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac299c-7a01-4b38-a2ac-5c9fd9c1d57b service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:55.216859 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac299c-7a01-4b38-a2ac-5c9fd9c1d57b service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:55.282399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ce678-c4ee-41d6-8e4e-7c912e35d272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:55.283178 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ce678-c4ee-41d6-8e4e-7c912e35d272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:55.283453 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ce678-c4ee-41d6-8e4e-7c912e35d272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:55.283723 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ce678-c4ee-41d6-8e4e-7c912e35d272 admin admin] PATH_INFO: `/v3/projects/0da9a82039c24056beec4b61f533eaa2` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:55.284929 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ce678-c4ee-41d6-8e4e-7c912e35d272 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0da9a82039c24056beec4b61f533eaa2)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:55.286817 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ce678-c4ee-41d6-8e4e-7c912e35d272 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:55.334517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-994ce678-c4ee-41d6-8e4e-7c912e35d272 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0da9a82039c24056beec4b61f533eaa2', 'request_id': 'req-994ce678-c4ee-41d6-8e4e-7c912e35d272'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:55.336609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:13:55.344253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-994ce678-c4ee-41d6-8e4e-7c912e35d272 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0da9a82039c24056beec4b61f533eaa2', 'request_id': 'req-994ce678-c4ee-41d6-8e4e-7c912e35d272'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:55.350381 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-994ce678-c4ee-41d6-8e4e-7c912e35d272 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0da9a82039c24056beec4b61f533eaa2', 'request_id': 'req-994ce678-c4ee-41d6-8e4e-7c912e35d272'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:55.363969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-994ce678-c4ee-41d6-8e4e-7c912e35d272 admin admin] The token cache is being invalidate because project 0da9a82039c24056beec4b61f533eaa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:13:55.364439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-994ce678-c4ee-41d6-8e4e-7c912e35d272 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:55.365195 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-994ce678-c4ee-41d6-8e4e-7c912e35d272 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:13:55.366964 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1312/2593] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:13:55 2022] DELETE /identity/v3/projects/0da9a82039c24056beec4b61f533eaa2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:13:55.372393 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3d39c-eb96-4a76-92b1-e53d4c00b877 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:55.372802 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3d39c-eb96-4a76-92b1-e53d4c00b877 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:55.373154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3d39c-eb96-4a76-92b1-e53d4c00b877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:55.392353 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-86e3d39c-eb96-4a76-92b1-e53d4c00b877 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:13:55.393610 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-86e3d39c-eb96-4a76-92b1-e53d4c00b877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:55.459206 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1282/2594] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:13:55.468179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa5677-710c-4c24-a548-5bfb8ac5b9aa None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:55.468529 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa5677-710c-4c24-a548-5bfb8ac5b9aa None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:55.468803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa5677-710c-4c24-a548-5bfb8ac5b9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:55.487861 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1faa5677-710c-4c24-a548-5bfb8ac5b9aa None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:13:55.489210 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1faa5677-710c-4c24-a548-5bfb8ac5b9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:55.497838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1313/2595] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:13:55.502103 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3d39c-eb96-4a76-92b1-e53d4c00b877 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:55.513483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3d39c-eb96-4a76-92b1-e53d4c00b877 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:55.519457 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309a493-268e-4257-80a7-98e213f8d04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:55.520126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309a493-268e-4257-80a7-98e213f8d04b admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:55.520432 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309a493-268e-4257-80a7-98e213f8d04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:55.520723 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a309a493-268e-4257-80a7-98e213f8d04b admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:55.521459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309a493-268e-4257-80a7-98e213f8d04b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:13:55.521766 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309a493-268e-4257-80a7-98e213f8d04b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:55.523391 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a309a493-268e-4257-80a7-98e213f8d04b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:55.535890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1283/2596] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:13:55.540066 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa5677-710c-4c24-a548-5bfb8ac5b9aa None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:55.550166 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa5677-710c-4c24-a548-5bfb8ac5b9aa None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:55.555235 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab185c89-fbe5-4c76-adb4-4b0848139bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:55.555879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab185c89-fbe5-4c76-adb4-4b0848139bea admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:55.556159 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab185c89-fbe5-4c76-adb4-4b0848139bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:55.556441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab185c89-fbe5-4c76-adb4-4b0848139bea admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:55.557140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab185c89-fbe5-4c76-adb4-4b0848139bea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:55.559062 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab185c89-fbe5-4c76-adb4-4b0848139bea admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:55.586600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1314/2597] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:13:55 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:13:55.590805 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309a493-268e-4257-80a7-98e213f8d04b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:55.601685 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a309a493-268e-4257-80a7-98e213f8d04b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:55.660517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106565b-e157-47ed-a125-9e6f47406789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:55.661184 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106565b-e157-47ed-a125-9e6f47406789 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:55.661458 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106565b-e157-47ed-a125-9e6f47406789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:55.661728 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106565b-e157-47ed-a125-9e6f47406789 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:55.662411 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106565b-e157-47ed-a125-9e6f47406789 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:55.664386 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106565b-e157-47ed-a125-9e6f47406789 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:55.671559 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:13:55.689134 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1284/2598] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:13:55 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:13:55.693185 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab185c89-fbe5-4c76-adb4-4b0848139bea admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:55.703903 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab185c89-fbe5-4c76-adb4-4b0848139bea admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:55.708696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9de7e0-7b61-4ea8-a429-3d6cd29f0495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:55.709355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9de7e0-7b61-4ea8-a429-3d6cd29f0495 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:55.709633 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9de7e0-7b61-4ea8-a429-3d6cd29f0495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:55.709904 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9de7e0-7b61-4ea8-a429-3d6cd29f0495 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:55.710625 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9de7e0-7b61-4ea8-a429-3d6cd29f0495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:55.712215 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9de7e0-7b61-4ea8-a429-3d6cd29f0495 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:55.722943 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1315/2599] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:13:55.726776 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106565b-e157-47ed-a125-9e6f47406789 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:55.737325 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106565b-e157-47ed-a125-9e6f47406789 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:55.742556 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce2125-b905-4608-b94a-eccc831b6f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:55.743200 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce2125-b905-4608-b94a-eccc831b6f72 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:55.743477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce2125-b905-4608-b94a-eccc831b6f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:55.743795 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce2125-b905-4608-b94a-eccc831b6f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:55.744543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce2125-b905-4608-b94a-eccc831b6f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:55.746166 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce2125-b905-4608-b94a-eccc831b6f72 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:55.756901 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1285/2600] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:13:55.761248 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9de7e0-7b61-4ea8-a429-3d6cd29f0495 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:55.772877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9de7e0-7b61-4ea8-a429-3d6cd29f0495 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:55.778594 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9804173-8af5-40d6-b35b-df77fc1c2e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:55.779505 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9804173-8af5-40d6-b35b-df77fc1c2e2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:55.779784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9804173-8af5-40d6-b35b-df77fc1c2e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:55.780058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9804173-8af5-40d6-b35b-df77fc1c2e2c admin admin] PATH_INFO: `/v3/projects/11871f54aa9446a4af64205fddabd9ca/users/57b7130aa84142dabbb4b056ab55479c/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:55.796615 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9804173-8af5-40d6-b35b-df77fc1c2e2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=11871f54aa9446a4af64205fddabd9ca, user_id=57b7130aa84142dabbb4b056ab55479c, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:55.799321 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9804173-8af5-40d6-b35b-df77fc1c2e2c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:55.815078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1316/2601] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:13:55 2022] PUT /identity/v3/projects/11871f54aa9446a4af64205fddabd9ca/users/57b7130aa84142dabbb4b056ab55479c/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:13:55.825666 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db0ce2-851a-44b9-8b16-ed812a6a227c None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:55.825929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db0ce2-851a-44b9-8b16-ed812a6a227c None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:55.826225 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db0ce2-851a-44b9-8b16-ed812a6a227c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:55.864357 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-87db0ce2-851a-44b9-8b16-ed812a6a227c None None] MFA Rules not processed for user `57b7130aa84142dabbb4b056ab55479c`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:13:55.865638 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-87db0ce2-851a-44b9-8b16-ed812a6a227c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:55.932636 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1286/2602] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:13:55 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:13:55.937407 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9804173-8af5-40d6-b35b-df77fc1c2e2c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:56.001601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9804173-8af5-40d6-b35b-df77fc1c2e2c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:56.006685 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321b215-8cb3-4905-a342-35c49bdd2b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:56.007346 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321b215-8cb3-4905-a342-35c49bdd2b42 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:56.007630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321b215-8cb3-4905-a342-35c49bdd2b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:56.007910 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321b215-8cb3-4905-a342-35c49bdd2b42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:56.008637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e321b215-8cb3-4905-a342-35c49bdd2b42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:56.010563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e321b215-8cb3-4905-a342-35c49bdd2b42 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:56.038985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1317/2603] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:13:55 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:13:56.049933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db0ce2-851a-44b9-8b16-ed812a6a227c None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:56.061942 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db0ce2-851a-44b9-8b16-ed812a6a227c None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:56.125976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f66e95-d338-4e56-8cc1-f019cbca230e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:56.126691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f66e95-d338-4e56-8cc1-f019cbca230e admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:56.126971 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f66e95-d338-4e56-8cc1-f019cbca230e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:56.127250 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f66e95-d338-4e56-8cc1-f019cbca230e admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:56.127945 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f66e95-d338-4e56-8cc1-f019cbca230e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:56.129917 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f66e95-d338-4e56-8cc1-f019cbca230e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:56.154301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1287/2604] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:13:56 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:13:56.159154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321b215-8cb3-4905-a342-35c49bdd2b42 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:56.170411 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321b215-8cb3-4905-a342-35c49bdd2b42 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:56.176508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629135a-53fe-4d37-b819-6e6dd42ab070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:56.177213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629135a-53fe-4d37-b819-6e6dd42ab070 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:56.177512 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629135a-53fe-4d37-b819-6e6dd42ab070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:56.177781 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629135a-53fe-4d37-b819-6e6dd42ab070 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:56.178501 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629135a-53fe-4d37-b819-6e6dd42ab070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:56.180125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629135a-53fe-4d37-b819-6e6dd42ab070 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:56.191289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1318/2605] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:13:56.197302 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f66e95-d338-4e56-8cc1-f019cbca230e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:56.207871 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f66e95-d338-4e56-8cc1-f019cbca230e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:56.214228 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2d4e6-9892-414a-867a-42a67891438e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:56.215163 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2d4e6-9892-414a-867a-42a67891438e admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:56.216471 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2d4e6-9892-414a-867a-42a67891438e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:56.216758 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2d4e6-9892-414a-867a-42a67891438e admin admin] PATH_INFO: `/v3/projects/9770d62e500646199b8d3839e08fa189/users/09a4cbbbf19c4e43896ec056683b8e39/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:56.232280 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2d4e6-9892-414a-867a-42a67891438e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9770d62e500646199b8d3839e08fa189, user_id=09a4cbbbf19c4e43896ec056683b8e39, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:56.235202 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2d4e6-9892-414a-867a-42a67891438e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:56.249778 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1288/2606] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:13:56 2022] PUT /identity/v3/projects/9770d62e500646199b8d3839e08fa189/users/09a4cbbbf19c4e43896ec056683b8e39/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:13:56.253802 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629135a-53fe-4d37-b819-6e6dd42ab070 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:56.264779 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629135a-53fe-4d37-b819-6e6dd42ab070 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:56.313677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:13:56.325435 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b586f-b901-4610-9231-bd50e1c0d781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:56.326089 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b586f-b901-4610-9231-bd50e1c0d781 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:56.326368 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b586f-b901-4610-9231-bd50e1c0d781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:56.326640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b586f-b901-4610-9231-bd50e1c0d781 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:56.327364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866b586f-b901-4610-9231-bd50e1c0d781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:56.329018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866b586f-b901-4610-9231-bd50e1c0d781 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:56.338639 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1319/2607] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:13:56.342889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2d4e6-9892-414a-867a-42a67891438e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:56.353877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2d4e6-9892-414a-867a-42a67891438e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:56.359522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b7509-a9f8-4866-bd02-51760bc1d0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:56.360447 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b7509-a9f8-4866-bd02-51760bc1d0f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:56.360748 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b7509-a9f8-4866-bd02-51760bc1d0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:56.361043 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b7509-a9f8-4866-bd02-51760bc1d0f4 admin admin] PATH_INFO: `/v3/projects/9770d62e500646199b8d3839e08fa189/users/09a4cbbbf19c4e43896ec056683b8e39/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:56.363236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b7509-a9f8-4866-bd02-51760bc1d0f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9770d62e500646199b8d3839e08fa189, user_id=09a4cbbbf19c4e43896ec056683b8e39, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:56.366033 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b7509-a9f8-4866-bd02-51760bc1d0f4 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:56.377635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1289/2608] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:13:56 2022] PUT /identity/v3/projects/9770d62e500646199b8d3839e08fa189/users/09a4cbbbf19c4e43896ec056683b8e39/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:13:56.387799 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec06d1-0b6a-459f-a577-23da623ff81b None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:56.388091 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec06d1-0b6a-459f-a577-23da623ff81b None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:56.388360 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec06d1-0b6a-459f-a577-23da623ff81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:56.416690 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-65ec06d1-0b6a-459f-a577-23da623ff81b None None] MFA Rules not processed for user `09a4cbbbf19c4e43896ec056683b8e39`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:13:56.418038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-65ec06d1-0b6a-459f-a577-23da623ff81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:56.491211 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1320/2609] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:13:56 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:13:56.504770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b7509-a9f8-4866-bd02-51760bc1d0f4 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:56.506800 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ee3b7509-a9f8-4866-bd02-51760bc1d0f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:56.534552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:13:56.571818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b7509-a9f8-4866-bd02-51760bc1d0f4 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:56.575793 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad431ec-7e9c-492a-9183-f8fccf8a8fdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:56.576560 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad431ec-7e9c-492a-9183-f8fccf8a8fdf service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:56.577087 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad431ec-7e9c-492a-9183-f8fccf8a8fdf service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:56.577423 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad431ec-7e9c-492a-9183-f8fccf8a8fdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:56.587086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad431ec-7e9c-492a-9183-f8fccf8a8fdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:56.588748 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad431ec-7e9c-492a-9183-f8fccf8a8fdf service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:56.598975 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1290/2610] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:13:56 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:13:56.676587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec06d1-0b6a-459f-a577-23da623ff81b None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:56.686814 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec06d1-0b6a-459f-a577-23da623ff81b None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:56.690335 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0933e2-fba1-4d90-a8cd-5eb76083351a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:56.690974 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0933e2-fba1-4d90-a8cd-5eb76083351a service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:56.691254 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0933e2-fba1-4d90-a8cd-5eb76083351a service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:56.691526 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0933e2-fba1-4d90-a8cd-5eb76083351a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:56.748030 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0933e2-fba1-4d90-a8cd-5eb76083351a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:56.749499 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0933e2-fba1-4d90-a8cd-5eb76083351a service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:56.758823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1321/2611] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:13:56 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:13:56.832583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad431ec-7e9c-492a-9183-f8fccf8a8fdf service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:13:56.834098 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bad431ec-7e9c-492a-9183-f8fccf8a8fdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:56.902720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad431ec-7e9c-492a-9183-f8fccf8a8fdf service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:13:56.910792 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da589b7b-a484-4659-a044-b4afb422f792 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:13:56.911472 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da589b7b-a484-4659-a044-b4afb422f792 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:13:56.911751 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da589b7b-a484-4659-a044-b4afb422f792 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:13:56.912025 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da589b7b-a484-4659-a044-b4afb422f792 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:13:56.913913 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-da589b7b-a484-4659-a044-b4afb422f792 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:13:56.979643 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da589b7b-a484-4659-a044-b4afb422f792 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:13:56.981400 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da589b7b-a484-4659-a044-b4afb422f792 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:13:56.990773 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1291/2612] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:13:56 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:14:06.092320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0933e2-fba1-4d90-a8cd-5eb76083351a service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:06.174580 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0933e2-fba1-4d90-a8cd-5eb76083351a service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:06.179661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42791f96-2ba4-46a9-870e-fed3b7a701a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:06.180590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42791f96-2ba4-46a9-870e-fed3b7a701a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:06.180860 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42791f96-2ba4-46a9-870e-fed3b7a701a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:06.181132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42791f96-2ba4-46a9-870e-fed3b7a701a2 admin admin] PATH_INFO: `/v3/users/57b7130aa84142dabbb4b056ab55479c` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:06.182394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42791f96-2ba4-46a9-870e-fed3b7a701a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57b7130aa84142dabbb4b056ab55479c)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:06.184529 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42791f96-2ba4-46a9-870e-fed3b7a701a2 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:06.197216 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:06.253275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-42791f96-2ba4-46a9-870e-fed3b7a701a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57b7130aa84142dabbb4b056ab55479c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:06.260780 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-42791f96-2ba4-46a9-870e-fed3b7a701a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57b7130aa84142dabbb4b056ab55479c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:06.261630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-42791f96-2ba4-46a9-870e-fed3b7a701a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57b7130aa84142dabbb4b056ab55479c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:06.272254 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-42791f96-2ba4-46a9-870e-fed3b7a701a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57b7130aa84142dabbb4b056ab55479c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:06.273025 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-42791f96-2ba4-46a9-870e-fed3b7a701a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57b7130aa84142dabbb4b056ab55479c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:06.293297 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1322/2613] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:14:06 2022] DELETE /identity/v3/users/57b7130aa84142dabbb4b056ab55479c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:06.297953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b8770-d628-4de8-803f-5f8d0a78a529 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:06.298439 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b8770-d628-4de8-803f-5f8d0a78a529 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:06.298537 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b8770-d628-4de8-803f-5f8d0a78a529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:06.316005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-dd7b8770-d628-4de8-803f-5f8d0a78a529 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:06.317235 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-dd7b8770-d628-4de8-803f-5f8d0a78a529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:06.381890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1292/2614] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:06.392628 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42791f96-2ba4-46a9-870e-fed3b7a701a2 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:06.395071 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-42791f96-2ba4-46a9-870e-fed3b7a701a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:06.463067 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42791f96-2ba4-46a9-870e-fed3b7a701a2 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:06.468306 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ace8e-616c-477a-9a26-850728344012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:06.468984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ace8e-616c-477a-9a26-850728344012 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:06.469261 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ace8e-616c-477a-9a26-850728344012 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:06.469531 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ace8e-616c-477a-9a26-850728344012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:06.479714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ace8e-616c-477a-9a26-850728344012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:06.481185 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ace8e-616c-477a-9a26-850728344012 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:06.490955 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1323/2615] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:14:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:14:06.792554 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b8770-d628-4de8-803f-5f8d0a78a529 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:06.794124 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-dd7b8770-d628-4de8-803f-5f8d0a78a529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:06.808433 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b8770-d628-4de8-803f-5f8d0a78a529 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:06.814081 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac7bf1-dbad-496f-be93-5e83480464ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:06.815110 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac7bf1-dbad-496f-be93-5e83480464ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:06.815467 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac7bf1-dbad-496f-be93-5e83480464ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:06.815738 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac7bf1-dbad-496f-be93-5e83480464ea admin admin] PATH_INFO: `/v3/projects/11871f54aa9446a4af64205fddabd9ca` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:06.817036 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac7bf1-dbad-496f-be93-5e83480464ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=11871f54aa9446a4af64205fddabd9ca)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:06.819141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac7bf1-dbad-496f-be93-5e83480464ea admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:06.828195 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:06.872783 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d6ac7bf1-dbad-496f-be93-5e83480464ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11871f54aa9446a4af64205fddabd9ca', 'request_id': 'req-d6ac7bf1-dbad-496f-be93-5e83480464ea'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:06.878817 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d6ac7bf1-dbad-496f-be93-5e83480464ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11871f54aa9446a4af64205fddabd9ca', 'request_id': 'req-d6ac7bf1-dbad-496f-be93-5e83480464ea'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:06.887821 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d6ac7bf1-dbad-496f-be93-5e83480464ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11871f54aa9446a4af64205fddabd9ca', 'request_id': 'req-d6ac7bf1-dbad-496f-be93-5e83480464ea'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:06.902839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d6ac7bf1-dbad-496f-be93-5e83480464ea admin admin] The token cache is being invalidate because project 11871f54aa9446a4af64205fddabd9ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:14:06.903052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d6ac7bf1-dbad-496f-be93-5e83480464ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:06.903710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d6ac7bf1-dbad-496f-be93-5e83480464ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:06.905349 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1293/2616] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:14:06 2022] DELETE /identity/v3/projects/11871f54aa9446a4af64205fddabd9ca => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:06.909567 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ace8e-616c-477a-9a26-850728344012 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:06.919564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ace8e-616c-477a-9a26-850728344012 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:06.977786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a583c05-77e1-4ae3-a408-bc57eced2e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:06.978665 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a583c05-77e1-4ae3-a408-bc57eced2e8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:06.978941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a583c05-77e1-4ae3-a408-bc57eced2e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:06.979222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a583c05-77e1-4ae3-a408-bc57eced2e8b admin admin] PATH_INFO: `/v3/users/09a4cbbbf19c4e43896ec056683b8e39` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:06.980520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a583c05-77e1-4ae3-a408-bc57eced2e8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=09a4cbbbf19c4e43896ec056683b8e39)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:06.982611 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a583c05-77e1-4ae3-a408-bc57eced2e8b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:07.024054 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:07.035371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9a583c05-77e1-4ae3-a408-bc57eced2e8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09a4cbbbf19c4e43896ec056683b8e39'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:07.042549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9a583c05-77e1-4ae3-a408-bc57eced2e8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09a4cbbbf19c4e43896ec056683b8e39'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:07.043858 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9a583c05-77e1-4ae3-a408-bc57eced2e8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09a4cbbbf19c4e43896ec056683b8e39'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:07.056252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9a583c05-77e1-4ae3-a408-bc57eced2e8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09a4cbbbf19c4e43896ec056683b8e39'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:07.057042 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9a583c05-77e1-4ae3-a408-bc57eced2e8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09a4cbbbf19c4e43896ec056683b8e39'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:07.077864 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1324/2617] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:14:06 2022] DELETE /identity/v3/users/09a4cbbbf19c4e43896ec056683b8e39 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:14:07.362697 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac7bf1-dbad-496f-be93-5e83480464ea admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:07.364318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d6ac7bf1-dbad-496f-be93-5e83480464ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:07.378549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac7bf1-dbad-496f-be93-5e83480464ea admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:07.438573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4b135-5315-4d8f-a1ba-10372062824c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:07.439474 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4b135-5315-4d8f-a1ba-10372062824c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:07.439757 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4b135-5315-4d8f-a1ba-10372062824c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:07.440052 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4b135-5315-4d8f-a1ba-10372062824c admin admin] PATH_INFO: `/v3/projects/9770d62e500646199b8d3839e08fa189` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:07.441338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4b135-5315-4d8f-a1ba-10372062824c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9770d62e500646199b8d3839e08fa189)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:07.443285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4b135-5315-4d8f-a1ba-10372062824c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:07.492660 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3ce4b135-5315-4d8f-a1ba-10372062824c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9770d62e500646199b8d3839e08fa189', 'request_id': 'req-3ce4b135-5315-4d8f-a1ba-10372062824c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:07.501450 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3ce4b135-5315-4d8f-a1ba-10372062824c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9770d62e500646199b8d3839e08fa189', 'request_id': 'req-3ce4b135-5315-4d8f-a1ba-10372062824c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:07.510474 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3ce4b135-5315-4d8f-a1ba-10372062824c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9770d62e500646199b8d3839e08fa189', 'request_id': 'req-3ce4b135-5315-4d8f-a1ba-10372062824c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:07.528673 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3ce4b135-5315-4d8f-a1ba-10372062824c admin admin] The token cache is being invalidate because project 9770d62e500646199b8d3839e08fa189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:14:07.528996 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3ce4b135-5315-4d8f-a1ba-10372062824c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:07.530045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3ce4b135-5315-4d8f-a1ba-10372062824c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:07.532431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1294/2618] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:14:07 2022] DELETE /identity/v3/projects/9770d62e500646199b8d3839e08fa189 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:07.538201 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d287e-3845-417c-aac9-2f62331150d7 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:07.538702 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d287e-3845-417c-aac9-2f62331150d7 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:07.538935 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d287e-3845-417c-aac9-2f62331150d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:07.558669 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-4d5d287e-3845-417c-aac9-2f62331150d7 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:07.559886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4d5d287e-3845-417c-aac9-2f62331150d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:07.629214 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1325/2619] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:07.641043 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01420d6f-e9d2-406a-a2e7-779a12158eb5 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:07.641418 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01420d6f-e9d2-406a-a2e7-779a12158eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:07.641731 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01420d6f-e9d2-406a-a2e7-779a12158eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:07.662090 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-01420d6f-e9d2-406a-a2e7-779a12158eb5 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:07.663301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-01420d6f-e9d2-406a-a2e7-779a12158eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:07.673058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1295/2620] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:07.677844 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d287e-3845-417c-aac9-2f62331150d7 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:07.691421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d287e-3845-417c-aac9-2f62331150d7 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:07.697716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3db0af-83b5-47be-bc13-4e7d3753a547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:07.698409 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3db0af-83b5-47be-bc13-4e7d3753a547 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:07.698692 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3db0af-83b5-47be-bc13-4e7d3753a547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:07.699116 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3db0af-83b5-47be-bc13-4e7d3753a547 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:07.699740 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3db0af-83b5-47be-bc13-4e7d3753a547 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:14:07.700046 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3db0af-83b5-47be-bc13-4e7d3753a547 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:07.701726 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3db0af-83b5-47be-bc13-4e7d3753a547 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:07.714186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1326/2621] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:14:07.718776 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01420d6f-e9d2-406a-a2e7-779a12158eb5 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:07.729033 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01420d6f-e9d2-406a-a2e7-779a12158eb5 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:07.734065 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269b459-9f72-4ce4-92a4-c453fea242ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:07.734734 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269b459-9f72-4ce4-92a4-c453fea242ea admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:07.735007 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269b459-9f72-4ce4-92a4-c453fea242ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:07.735276 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269b459-9f72-4ce4-92a4-c453fea242ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:07.735965 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269b459-9f72-4ce4-92a4-c453fea242ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:07.737918 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269b459-9f72-4ce4-92a4-c453fea242ea admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:07.758050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:07.773185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1296/2622] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:14:07 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:14:07.774409 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3db0af-83b5-47be-bc13-4e7d3753a547 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:07.785134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3db0af-83b5-47be-bc13-4e7d3753a547 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:07.843460 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ece1e-1411-47d2-ac48-428e39a43a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:07.844104 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ece1e-1411-47d2-ac48-428e39a43a75 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:07.844399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ece1e-1411-47d2-ac48-428e39a43a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:07.844687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ece1e-1411-47d2-ac48-428e39a43a75 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:07.845384 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ece1e-1411-47d2-ac48-428e39a43a75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:07.847297 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ece1e-1411-47d2-ac48-428e39a43a75 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:07.867996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1327/2623] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:14:07 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:14:07.872287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269b459-9f72-4ce4-92a4-c453fea242ea admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:07.882965 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269b459-9f72-4ce4-92a4-c453fea242ea admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:07.888045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9072e4d-dc69-4af5-8f1e-bb16e92494a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:07.888720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9072e4d-dc69-4af5-8f1e-bb16e92494a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:07.889325 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9072e4d-dc69-4af5-8f1e-bb16e92494a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:07.889598 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9072e4d-dc69-4af5-8f1e-bb16e92494a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:07.890305 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9072e4d-dc69-4af5-8f1e-bb16e92494a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:07.891906 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9072e4d-dc69-4af5-8f1e-bb16e92494a1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:07.902418 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1297/2624] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:14:07.906522 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ece1e-1411-47d2-ac48-428e39a43a75 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:07.916858 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ece1e-1411-47d2-ac48-428e39a43a75 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:07.922314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23f8fc-b81b-4f3b-8f3f-1057a8e664b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:07.923391 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23f8fc-b81b-4f3b-8f3f-1057a8e664b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:07.923679 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23f8fc-b81b-4f3b-8f3f-1057a8e664b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:07.923976 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23f8fc-b81b-4f3b-8f3f-1057a8e664b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:07.924779 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23f8fc-b81b-4f3b-8f3f-1057a8e664b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:07.926370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23f8fc-b81b-4f3b-8f3f-1057a8e664b1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:07.937408 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1328/2625] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:14:07.941717 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9072e4d-dc69-4af5-8f1e-bb16e92494a1 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:07.958049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9072e4d-dc69-4af5-8f1e-bb16e92494a1 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:07.963552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501c974-93a4-4bb0-b582-ff4e9f64e8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:07.964456 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501c974-93a4-4bb0-b582-ff4e9f64e8a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:07.964765 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501c974-93a4-4bb0-b582-ff4e9f64e8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:07.965061 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501c974-93a4-4bb0-b582-ff4e9f64e8a8 admin admin] PATH_INFO: `/v3/projects/119fb0d40ac74365ba9646c230ace5f4/users/d6f4f4d935734d2fb02284320ce9be12/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:07.987331 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501c974-93a4-4bb0-b582-ff4e9f64e8a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=119fb0d40ac74365ba9646c230ace5f4, user_id=d6f4f4d935734d2fb02284320ce9be12, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:07.990413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c501c974-93a4-4bb0-b582-ff4e9f64e8a8 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:08.006444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1298/2626] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:14:07 2022] PUT /identity/v3/projects/119fb0d40ac74365ba9646c230ace5f4/users/d6f4f4d935734d2fb02284320ce9be12/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:08.017674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da478c54-9e2c-4ea4-aec0-f23988e74e24 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:08.017965 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da478c54-9e2c-4ea4-aec0-f23988e74e24 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:08.018245 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da478c54-9e2c-4ea4-aec0-f23988e74e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:08.045875 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-da478c54-9e2c-4ea4-aec0-f23988e74e24 None None] MFA Rules not processed for user `d6f4f4d935734d2fb02284320ce9be12`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:08.047070 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-da478c54-9e2c-4ea4-aec0-f23988e74e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:08.052253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:08.119069 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1329/2627] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:14:08.124599 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501c974-93a4-4bb0-b582-ff4e9f64e8a8 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:08.190890 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c501c974-93a4-4bb0-b582-ff4e9f64e8a8 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:08.200175 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae0e86-4825-4f3d-a0f3-7ba1d9e8a6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:08.200910 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae0e86-4825-4f3d-a0f3-7ba1d9e8a6a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:08.201192 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae0e86-4825-4f3d-a0f3-7ba1d9e8a6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:08.201467 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae0e86-4825-4f3d-a0f3-7ba1d9e8a6a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:08.202169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae0e86-4825-4f3d-a0f3-7ba1d9e8a6a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:08.204128 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae0e86-4825-4f3d-a0f3-7ba1d9e8a6a5 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:08.233248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1299/2628] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:14:08 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:14:08.238717 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da478c54-9e2c-4ea4-aec0-f23988e74e24 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:08.254621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da478c54-9e2c-4ea4-aec0-f23988e74e24 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:08.320426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e314444-cf87-4706-bd12-4bf06bc8f185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:08.321116 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e314444-cf87-4706-bd12-4bf06bc8f185 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:08.321399 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e314444-cf87-4706-bd12-4bf06bc8f185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:08.321674 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e314444-cf87-4706-bd12-4bf06bc8f185 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:08.322382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e314444-cf87-4706-bd12-4bf06bc8f185 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:08.324326 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e314444-cf87-4706-bd12-4bf06bc8f185 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:08.354309 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1330/2629] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:14:08 2022] POST /identity/v3/users => generated 297 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:14:08.356100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae0e86-4825-4f3d-a0f3-7ba1d9e8a6a5 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:08.367238 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae0e86-4825-4f3d-a0f3-7ba1d9e8a6a5 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:08.372657 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b1c4e-de8b-44d0-b7bc-1164a7fadee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:08.373313 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b1c4e-de8b-44d0-b7bc-1164a7fadee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:08.373590 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b1c4e-de8b-44d0-b7bc-1164a7fadee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:08.373856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b1c4e-de8b-44d0-b7bc-1164a7fadee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:08.374573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b1c4e-de8b-44d0-b7bc-1164a7fadee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:08.376202 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b1c4e-de8b-44d0-b7bc-1164a7fadee6 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:08.387162 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1300/2630] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:14:08.392983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e314444-cf87-4706-bd12-4bf06bc8f185 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:08.403586 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e314444-cf87-4706-bd12-4bf06bc8f185 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:08.409151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaaf880-ec77-418a-80e3-f875e48925cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:08.410027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaf880-ec77-418a-80e3-f875e48925cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:08.410304 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaf880-ec77-418a-80e3-f875e48925cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:08.410579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaaf880-ec77-418a-80e3-f875e48925cc admin admin] PATH_INFO: `/v3/projects/f11a6d84103b475189c21c1cb6a12bf6/users/178833e159634e63a717c01ec9935c20/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:08.427185 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaaf880-ec77-418a-80e3-f875e48925cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f11a6d84103b475189c21c1cb6a12bf6, user_id=178833e159634e63a717c01ec9935c20, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:08.429869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaaf880-ec77-418a-80e3-f875e48925cc admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:08.444876 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1331/2631] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:14:08 2022] PUT /identity/v3/projects/f11a6d84103b475189c21c1cb6a12bf6/users/178833e159634e63a717c01ec9935c20/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:08.448814 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b1c4e-de8b-44d0-b7bc-1164a7fadee6 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:08.459060 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b1c4e-de8b-44d0-b7bc-1164a7fadee6 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:08.522806 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c8b3f-be5e-4eb0-982e-ecf76002e3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:08.523483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c8b3f-be5e-4eb0-982e-ecf76002e3ba admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:08.523829 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c8b3f-be5e-4eb0-982e-ecf76002e3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:08.524189 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c8b3f-be5e-4eb0-982e-ecf76002e3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:08.524968 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c8b3f-be5e-4eb0-982e-ecf76002e3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:08.526752 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c8b3f-be5e-4eb0-982e-ecf76002e3ba admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:08.536904 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1301/2632] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:14:08.540889 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaaf880-ec77-418a-80e3-f875e48925cc admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:08.551512 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaaf880-ec77-418a-80e3-f875e48925cc admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:08.556507 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad29e7f-b1af-4c71-a8a0-519a7816301c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:08.557354 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad29e7f-b1af-4c71-a8a0-519a7816301c admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:08.557637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad29e7f-b1af-4c71-a8a0-519a7816301c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:08.557916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad29e7f-b1af-4c71-a8a0-519a7816301c admin admin] PATH_INFO: `/v3/projects/f11a6d84103b475189c21c1cb6a12bf6/users/178833e159634e63a717c01ec9935c20/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:08.559914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad29e7f-b1af-4c71-a8a0-519a7816301c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f11a6d84103b475189c21c1cb6a12bf6, user_id=178833e159634e63a717c01ec9935c20, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:08.562588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad29e7f-b1af-4c71-a8a0-519a7816301c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:08.573132 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1332/2633] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:14:08 2022] PUT /identity/v3/projects/f11a6d84103b475189c21c1cb6a12bf6/users/178833e159634e63a717c01ec9935c20/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:08.584458 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4907f-60b4-46af-8a40-1010457370e1 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:08.584764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4907f-60b4-46af-8a40-1010457370e1 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:08.585041 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4907f-60b4-46af-8a40-1010457370e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:08.618158 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-02a4907f-60b4-46af-8a40-1010457370e1 None None] MFA Rules not processed for user `178833e159634e63a717c01ec9935c20`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:08.619928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-02a4907f-60b4-46af-8a40-1010457370e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:08.643791 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:08.698798 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1302/2634] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:14:08.710209 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad29e7f-b1af-4c71-a8a0-519a7816301c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:08.711977 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-dad29e7f-b1af-4c71-a8a0-519a7816301c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:08.776788 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad29e7f-b1af-4c71-a8a0-519a7816301c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:08.780996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbaf6a-746f-4264-97af-08779caa218f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:08.781680 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbaf6a-746f-4264-97af-08779caa218f service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:08.781985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbaf6a-746f-4264-97af-08779caa218f service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:08.782257 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbaf6a-746f-4264-97af-08779caa218f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:08.792139 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbaf6a-746f-4264-97af-08779caa218f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:08.793683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbaf6a-746f-4264-97af-08779caa218f service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:08.803508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1333/2635] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:14:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:14:09.238944 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4907f-60b4-46af-8a40-1010457370e1 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:09.309539 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4907f-60b4-46af-8a40-1010457370e1 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:09.315561 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb7b3b-b46e-4d54-8829-df0dff6e7274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:09.316552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb7b3b-b46e-4d54-8829-df0dff6e7274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:09.317138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb7b3b-b46e-4d54-8829-df0dff6e7274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:09.317489 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb7b3b-b46e-4d54-8829-df0dff6e7274 admin admin] PATH_INFO: `/v3/users/d6f4f4d935734d2fb02284320ce9be12` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:09.319031 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb7b3b-b46e-4d54-8829-df0dff6e7274 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6f4f4d935734d2fb02284320ce9be12)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:09.321312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb7b3b-b46e-4d54-8829-df0dff6e7274 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:09.373186 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e8bb7b3b-b46e-4d54-8829-df0dff6e7274 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6f4f4d935734d2fb02284320ce9be12'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:09.373885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e8bb7b3b-b46e-4d54-8829-df0dff6e7274 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6f4f4d935734d2fb02284320ce9be12'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:09.383040 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e8bb7b3b-b46e-4d54-8829-df0dff6e7274 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6f4f4d935734d2fb02284320ce9be12'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:09.391989 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e8bb7b3b-b46e-4d54-8829-df0dff6e7274 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6f4f4d935734d2fb02284320ce9be12'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:09.392694 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e8bb7b3b-b46e-4d54-8829-df0dff6e7274 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6f4f4d935734d2fb02284320ce9be12'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:09.411629 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1303/2636] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:14:09 2022] DELETE /identity/v3/users/d6f4f4d935734d2fb02284320ce9be12 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:09.416494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437f07f-3a6e-49cc-80a8-8b21cf2f00a1 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:09.416779 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437f07f-3a6e-49cc-80a8-8b21cf2f00a1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:09.417045 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437f07f-3a6e-49cc-80a8-8b21cf2f00a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:09.423902 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:09.437963 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7437f07f-3a6e-49cc-80a8-8b21cf2f00a1 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:09.439153 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7437f07f-3a6e-49cc-80a8-8b21cf2f00a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:09.504597 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1334/2637] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:09.515571 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb7b3b-b46e-4d54-8829-df0dff6e7274 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:09.517241 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e8bb7b3b-b46e-4d54-8829-df0dff6e7274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:09.560930 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:09.580172 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb7b3b-b46e-4d54-8829-df0dff6e7274 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:09.584333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5735d-9c3a-4ece-a0f0-a92c9f236c03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:09.585015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5735d-9c3a-4ece-a0f0-a92c9f236c03 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:09.585668 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5735d-9c3a-4ece-a0f0-a92c9f236c03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:09.585668 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5735d-9c3a-4ece-a0f0-a92c9f236c03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:09.595603 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb5735d-9c3a-4ece-a0f0-a92c9f236c03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:09.597092 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb5735d-9c3a-4ece-a0f0-a92c9f236c03 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:09.606622 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1304/2638] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:14:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:14:09.900083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437f07f-3a6e-49cc-80a8-8b21cf2f00a1 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:09.902177 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7437f07f-3a6e-49cc-80a8-8b21cf2f00a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:09.924122 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437f07f-3a6e-49cc-80a8-8b21cf2f00a1 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:09.929434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db6a41-b1f3-4c17-be4a-064175a80f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:09.930318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db6a41-b1f3-4c17-be4a-064175a80f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:09.930601 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db6a41-b1f3-4c17-be4a-064175a80f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:09.930971 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db6a41-b1f3-4c17-be4a-064175a80f11 admin admin] PATH_INFO: `/v3/projects/119fb0d40ac74365ba9646c230ace5f4` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:09.932066 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db6a41-b1f3-4c17-be4a-064175a80f11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=119fb0d40ac74365ba9646c230ace5f4)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:09.934151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db6a41-b1f3-4c17-be4a-064175a80f11 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:09.988369 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e1db6a41-b1f3-4c17-be4a-064175a80f11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '119fb0d40ac74365ba9646c230ace5f4', 'request_id': 'req-e1db6a41-b1f3-4c17-be4a-064175a80f11'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:10.000017 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e1db6a41-b1f3-4c17-be4a-064175a80f11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '119fb0d40ac74365ba9646c230ace5f4', 'request_id': 'req-e1db6a41-b1f3-4c17-be4a-064175a80f11'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:10.006637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e1db6a41-b1f3-4c17-be4a-064175a80f11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '119fb0d40ac74365ba9646c230ace5f4', 'request_id': 'req-e1db6a41-b1f3-4c17-be4a-064175a80f11'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:10.021248 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e1db6a41-b1f3-4c17-be4a-064175a80f11 admin admin] The token cache is being invalidate because project 119fb0d40ac74365ba9646c230ace5f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:14:10.021533 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e1db6a41-b1f3-4c17-be4a-064175a80f11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:10.022182 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e1db6a41-b1f3-4c17-be4a-064175a80f11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:10.023769 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1335/2639] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:14:09 2022] DELETE /identity/v3/projects/119fb0d40ac74365ba9646c230ace5f4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:10.028155 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5735d-9c3a-4ece-a0f0-a92c9f236c03 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:10.038311 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5735d-9c3a-4ece-a0f0-a92c9f236c03 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:10.098185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564c526-cb59-4868-8872-1f6e51d2e6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:10.099058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564c526-cb59-4868-8872-1f6e51d2e6de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:10.099336 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564c526-cb59-4868-8872-1f6e51d2e6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:10.099611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564c526-cb59-4868-8872-1f6e51d2e6de admin admin] PATH_INFO: `/v3/users/178833e159634e63a717c01ec9935c20` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:10.101025 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564c526-cb59-4868-8872-1f6e51d2e6de admin admin] RBAC: Authorizing `identity:delete_user(user_id=178833e159634e63a717c01ec9935c20)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:10.102951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564c526-cb59-4868-8872-1f6e51d2e6de admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:10.157661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-2564c526-cb59-4868-8872-1f6e51d2e6de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '178833e159634e63a717c01ec9935c20'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:10.158413 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-2564c526-cb59-4868-8872-1f6e51d2e6de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '178833e159634e63a717c01ec9935c20'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:10.165757 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-2564c526-cb59-4868-8872-1f6e51d2e6de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '178833e159634e63a717c01ec9935c20'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:10.172993 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-2564c526-cb59-4868-8872-1f6e51d2e6de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '178833e159634e63a717c01ec9935c20'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:10.173644 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-2564c526-cb59-4868-8872-1f6e51d2e6de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '178833e159634e63a717c01ec9935c20'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:10.191154 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1305/2640] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:14:10 2022] DELETE /identity/v3/users/178833e159634e63a717c01ec9935c20 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:10.474202 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db6a41-b1f3-4c17-be4a-064175a80f11 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:10.475782 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e1db6a41-b1f3-4c17-be4a-064175a80f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:10.491209 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db6a41-b1f3-4c17-be4a-064175a80f11 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:10.554807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452755e0-58e5-485f-805e-040603062c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:10.556001 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452755e0-58e5-485f-805e-040603062c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:10.556439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452755e0-58e5-485f-805e-040603062c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:10.556892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452755e0-58e5-485f-805e-040603062c2c admin admin] PATH_INFO: `/v3/projects/f11a6d84103b475189c21c1cb6a12bf6` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:10.558392 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452755e0-58e5-485f-805e-040603062c2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f11a6d84103b475189c21c1cb6a12bf6)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:10.560355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452755e0-58e5-485f-805e-040603062c2c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:10.593870 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:10.609830 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-452755e0-58e5-485f-805e-040603062c2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f11a6d84103b475189c21c1cb6a12bf6', 'request_id': 'req-452755e0-58e5-485f-805e-040603062c2c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:10.615092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-452755e0-58e5-485f-805e-040603062c2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f11a6d84103b475189c21c1cb6a12bf6', 'request_id': 'req-452755e0-58e5-485f-805e-040603062c2c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:10.621290 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-452755e0-58e5-485f-805e-040603062c2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f11a6d84103b475189c21c1cb6a12bf6', 'request_id': 'req-452755e0-58e5-485f-805e-040603062c2c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:10.634585 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-452755e0-58e5-485f-805e-040603062c2c admin admin] The token cache is being invalidate because project f11a6d84103b475189c21c1cb6a12bf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:14:10.634985 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-452755e0-58e5-485f-805e-040603062c2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:10.635600 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-452755e0-58e5-485f-805e-040603062c2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:10.637318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1336/2641] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:14:10 2022] DELETE /identity/v3/projects/f11a6d84103b475189c21c1cb6a12bf6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:10.642563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d310e-4f75-4d91-842b-3e56dddbc4f6 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:10.643034 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d310e-4f75-4d91-842b-3e56dddbc4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:10.643129 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d310e-4f75-4d91-842b-3e56dddbc4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:10.660255 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d20d310e-4f75-4d91-842b-3e56dddbc4f6 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:10.661513 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d20d310e-4f75-4d91-842b-3e56dddbc4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:10.718440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:10.733303 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1306/2642] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:10.737570 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078dd16-5368-481a-bb4b-983cf83007ab None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:10.737857 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078dd16-5368-481a-bb4b-983cf83007ab None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:10.738133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078dd16-5368-481a-bb4b-983cf83007ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:10.757004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-e078dd16-5368-481a-bb4b-983cf83007ab None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:10.758200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e078dd16-5368-481a-bb4b-983cf83007ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:10.767206 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1337/2643] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:10.771651 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d310e-4f75-4d91-842b-3e56dddbc4f6 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:10.782029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d310e-4f75-4d91-842b-3e56dddbc4f6 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:10.787227 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407be43-dd8d-484e-98e1-aafd914d3384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:10.787873 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407be43-dd8d-484e-98e1-aafd914d3384 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:10.788149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407be43-dd8d-484e-98e1-aafd914d3384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:10.788425 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407be43-dd8d-484e-98e1-aafd914d3384 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:10.789159 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407be43-dd8d-484e-98e1-aafd914d3384 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:14:10.789459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407be43-dd8d-484e-98e1-aafd914d3384 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:10.791121 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407be43-dd8d-484e-98e1-aafd914d3384 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:10.803062 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1307/2644] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:14:10.807640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078dd16-5368-481a-bb4b-983cf83007ab None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:10.817603 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078dd16-5368-481a-bb4b-983cf83007ab None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:10.823315 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716d229-4d13-46b5-8584-100c5c2fdf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:10.824045 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716d229-4d13-46b5-8584-100c5c2fdf95 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:10.824314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716d229-4d13-46b5-8584-100c5c2fdf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:10.824612 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716d229-4d13-46b5-8584-100c5c2fdf95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:10.825310 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c716d229-4d13-46b5-8584-100c5c2fdf95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:10.827239 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c716d229-4d13-46b5-8584-100c5c2fdf95 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:10.855915 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1338/2645] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:14:10 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:14:10.860364 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407be43-dd8d-484e-98e1-aafd914d3384 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:10.870973 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407be43-dd8d-484e-98e1-aafd914d3384 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:10.928758 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8091e6e-a824-4a56-94bd-3a5abe04b780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:10.929461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8091e6e-a824-4a56-94bd-3a5abe04b780 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:10.929742 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8091e6e-a824-4a56-94bd-3a5abe04b780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:10.930018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8091e6e-a824-4a56-94bd-3a5abe04b780 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:10.930799 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8091e6e-a824-4a56-94bd-3a5abe04b780 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:10.932895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8091e6e-a824-4a56-94bd-3a5abe04b780 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:10.953285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1308/2646] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:14:10 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:14:10.957314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716d229-4d13-46b5-8584-100c5c2fdf95 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:10.967937 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716d229-4d13-46b5-8584-100c5c2fdf95 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:10.972877 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdbf46-4cc4-43bd-9b56-c7ba79efe584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:10.973530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdbf46-4cc4-43bd-9b56-c7ba79efe584 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:10.973805 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdbf46-4cc4-43bd-9b56-c7ba79efe584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:10.974081 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdbf46-4cc4-43bd-9b56-c7ba79efe584 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:10.974810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fdbf46-4cc4-43bd-9b56-c7ba79efe584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:10.976395 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fdbf46-4cc4-43bd-9b56-c7ba79efe584 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:10.987644 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1339/2647] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:14:10.991770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8091e6e-a824-4a56-94bd-3a5abe04b780 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:11.003081 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8091e6e-a824-4a56-94bd-3a5abe04b780 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:11.008216 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ce108-c8c7-44e4-ab52-c1a2f21bdc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:11.008918 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ce108-c8c7-44e4-ab52-c1a2f21bdc7b admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:11.009199 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ce108-c8c7-44e4-ab52-c1a2f21bdc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:11.009512 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ce108-c8c7-44e4-ab52-c1a2f21bdc7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:11.010247 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ce108-c8c7-44e4-ab52-c1a2f21bdc7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:11.012279 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ce108-c8c7-44e4-ab52-c1a2f21bdc7b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:11.024045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1309/2648] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:14:11.028355 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdbf46-4cc4-43bd-9b56-c7ba79efe584 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:11.039015 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdbf46-4cc4-43bd-9b56-c7ba79efe584 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:11.043946 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f35bcf-49b5-432f-933a-3844fc4ca01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:11.044869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f35bcf-49b5-432f-933a-3844fc4ca01d admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:11.045147 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f35bcf-49b5-432f-933a-3844fc4ca01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:11.045420 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f35bcf-49b5-432f-933a-3844fc4ca01d admin admin] PATH_INFO: `/v3/projects/786c5efc8e624210a9ff93dee3e672e7/users/feadb88d0e604d8185609677ca20480e/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:11.061490 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f35bcf-49b5-432f-933a-3844fc4ca01d admin admin] RBAC: Authorizing `identity:create_grant(project_id=786c5efc8e624210a9ff93dee3e672e7, user_id=feadb88d0e604d8185609677ca20480e, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:11.064114 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f35bcf-49b5-432f-933a-3844fc4ca01d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:11.079113 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1340/2649] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:14:11 2022] PUT /identity/v3/projects/786c5efc8e624210a9ff93dee3e672e7/users/feadb88d0e604d8185609677ca20480e/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:11.089659 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679b0f3-3d48-4b99-ac92-c25def1ebafa None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:11.089947 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679b0f3-3d48-4b99-ac92-c25def1ebafa None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:11.090223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679b0f3-3d48-4b99-ac92-c25def1ebafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:11.116904 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d679b0f3-3d48-4b99-ac92-c25def1ebafa None None] MFA Rules not processed for user `feadb88d0e604d8185609677ca20480e`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:11.118144 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d679b0f3-3d48-4b99-ac92-c25def1ebafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:11.191375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1310/2650] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:11.203501 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f35bcf-49b5-432f-933a-3844fc4ca01d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:11.205383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-98f35bcf-49b5-432f-933a-3844fc4ca01d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:11.267955 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f35bcf-49b5-432f-933a-3844fc4ca01d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:11.272444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced59f94-25a5-4e3a-9f62-cf203acf892d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:11.273100 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced59f94-25a5-4e3a-9f62-cf203acf892d service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:11.273371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced59f94-25a5-4e3a-9f62-cf203acf892d service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:11.273649 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced59f94-25a5-4e3a-9f62-cf203acf892d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:11.282701 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced59f94-25a5-4e3a-9f62-cf203acf892d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:11.284106 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced59f94-25a5-4e3a-9f62-cf203acf892d service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:11.293361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1341/2651] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:14:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:14:11.313314 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679b0f3-3d48-4b99-ac92-c25def1ebafa None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:11.379761 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:11.386568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679b0f3-3d48-4b99-ac92-c25def1ebafa None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:11.392360 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58837af-d73d-437e-bded-d8adb88dbf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:11.393321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58837af-d73d-437e-bded-d8adb88dbf36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:11.393599 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58837af-d73d-437e-bded-d8adb88dbf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:11.393868 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58837af-d73d-437e-bded-d8adb88dbf36 admin admin] PATH_INFO: `/v3/users/feadb88d0e604d8185609677ca20480e` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:11.395161 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58837af-d73d-437e-bded-d8adb88dbf36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feadb88d0e604d8185609677ca20480e)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:11.397101 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58837af-d73d-437e-bded-d8adb88dbf36 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:11.445414 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e58837af-d73d-437e-bded-d8adb88dbf36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feadb88d0e604d8185609677ca20480e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:11.446092 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e58837af-d73d-437e-bded-d8adb88dbf36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feadb88d0e604d8185609677ca20480e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:11.454179 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e58837af-d73d-437e-bded-d8adb88dbf36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feadb88d0e604d8185609677ca20480e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:11.460995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e58837af-d73d-437e-bded-d8adb88dbf36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feadb88d0e604d8185609677ca20480e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:11.461607 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-e58837af-d73d-437e-bded-d8adb88dbf36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feadb88d0e604d8185609677ca20480e'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:11.478240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1311/2652] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:14:11 2022] DELETE /identity/v3/users/feadb88d0e604d8185609677ca20480e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:11.482810 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71feed-a690-4197-bf21-eb56948a42d1 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:11.483095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71feed-a690-4197-bf21-eb56948a42d1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:11.483373 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71feed-a690-4197-bf21-eb56948a42d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:11.500151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-af71feed-a690-4197-bf21-eb56948a42d1 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:11.501387 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-af71feed-a690-4197-bf21-eb56948a42d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:11.551521 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:11.565700 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1342/2653] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:11.577169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58837af-d73d-437e-bded-d8adb88dbf36 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:11.578755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e58837af-d73d-437e-bded-d8adb88dbf36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:11.645782 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58837af-d73d-437e-bded-d8adb88dbf36 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:11.649903 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16b541-d74b-4f84-bf81-f475becedbf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:11.650567 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16b541-d74b-4f84-bf81-f475becedbf6 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:11.650843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16b541-d74b-4f84-bf81-f475becedbf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:11.651089 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16b541-d74b-4f84-bf81-f475becedbf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:11.661113 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16b541-d74b-4f84-bf81-f475becedbf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:11.662544 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16b541-d74b-4f84-bf81-f475becedbf6 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:11.672452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1312/2654] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:14:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:14:11.944142 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71feed-a690-4197-bf21-eb56948a42d1 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:11.945604 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-af71feed-a690-4197-bf21-eb56948a42d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:11.959638 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71feed-a690-4197-bf21-eb56948a42d1 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:11.965485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a5cf6-2edf-45c8-82b1-0419663c3159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:11.966421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a5cf6-2edf-45c8-82b1-0419663c3159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:11.966649 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a5cf6-2edf-45c8-82b1-0419663c3159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:11.966926 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a5cf6-2edf-45c8-82b1-0419663c3159 admin admin] PATH_INFO: `/v3/projects/786c5efc8e624210a9ff93dee3e672e7` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:11.968318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a5cf6-2edf-45c8-82b1-0419663c3159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=786c5efc8e624210a9ff93dee3e672e7)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:11.970297 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a5cf6-2edf-45c8-82b1-0419663c3159 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:12.023908 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4d5a5cf6-2edf-45c8-82b1-0419663c3159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '786c5efc8e624210a9ff93dee3e672e7', 'request_id': 'req-4d5a5cf6-2edf-45c8-82b1-0419663c3159'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:12.030837 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4d5a5cf6-2edf-45c8-82b1-0419663c3159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '786c5efc8e624210a9ff93dee3e672e7', 'request_id': 'req-4d5a5cf6-2edf-45c8-82b1-0419663c3159'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:12.037940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4d5a5cf6-2edf-45c8-82b1-0419663c3159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '786c5efc8e624210a9ff93dee3e672e7', 'request_id': 'req-4d5a5cf6-2edf-45c8-82b1-0419663c3159'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:12.052709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4d5a5cf6-2edf-45c8-82b1-0419663c3159 admin admin] The token cache is being invalidate because project 786c5efc8e624210a9ff93dee3e672e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:14:12.052991 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4d5a5cf6-2edf-45c8-82b1-0419663c3159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:12.053597 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4d5a5cf6-2edf-45c8-82b1-0419663c3159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:12.055158 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1343/2655] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:14:11 2022] DELETE /identity/v3/projects/786c5efc8e624210a9ff93dee3e672e7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:12.060900 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d01280-e6d8-4b45-9f88-cfbc61ec3a05 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:12.061227 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d01280-e6d8-4b45-9f88-cfbc61ec3a05 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:12.061609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d01280-e6d8-4b45-9f88-cfbc61ec3a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:12.083412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-80d01280-e6d8-4b45-9f88-cfbc61ec3a05 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:12.084608 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-80d01280-e6d8-4b45-9f88-cfbc61ec3a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:12.151508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1313/2656] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:14:12.160530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c55fdc-686d-4353-8385-423c9c1f7741 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:12.160845 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c55fdc-686d-4353-8385-423c9c1f7741 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:12.161118 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c55fdc-686d-4353-8385-423c9c1f7741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:12.181198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-87c55fdc-686d-4353-8385-423c9c1f7741 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:12.182396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-87c55fdc-686d-4353-8385-423c9c1f7741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:12.191233 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1344/2657] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:12.195654 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d01280-e6d8-4b45-9f88-cfbc61ec3a05 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:12.205981 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d01280-e6d8-4b45-9f88-cfbc61ec3a05 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:12.210900 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97eb9d-872c-46a5-b55b-ebbba72b8643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:12.211552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97eb9d-872c-46a5-b55b-ebbba72b8643 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:12.211834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97eb9d-872c-46a5-b55b-ebbba72b8643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:12.212109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97eb9d-872c-46a5-b55b-ebbba72b8643 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:12.213096 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97eb9d-872c-46a5-b55b-ebbba72b8643 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:14:12.213399 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97eb9d-872c-46a5-b55b-ebbba72b8643 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:12.215126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97eb9d-872c-46a5-b55b-ebbba72b8643 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:12.227019 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1314/2658] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:14:12.231487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c55fdc-686d-4353-8385-423c9c1f7741 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:12.240906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c55fdc-686d-4353-8385-423c9c1f7741 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:12.246154 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ef144-1bde-43cb-9a70-16ffa010d87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:12.246983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ef144-1bde-43cb-9a70-16ffa010d87e admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:12.247269 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ef144-1bde-43cb-9a70-16ffa010d87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:12.247537 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ef144-1bde-43cb-9a70-16ffa010d87e admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:12.248224 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ef144-1bde-43cb-9a70-16ffa010d87e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:12.250120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ef144-1bde-43cb-9a70-16ffa010d87e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:12.278920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1345/2659] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:14:12 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:14:12.283542 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97eb9d-872c-46a5-b55b-ebbba72b8643 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:12.293417 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:12.297526 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97eb9d-872c-46a5-b55b-ebbba72b8643 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:12.367092 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc42b90-52dc-47ad-883a-6d0388eb4145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:12.367763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc42b90-52dc-47ad-883a-6d0388eb4145 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:12.368025 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc42b90-52dc-47ad-883a-6d0388eb4145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:12.368318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc42b90-52dc-47ad-883a-6d0388eb4145 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:12.369042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc42b90-52dc-47ad-883a-6d0388eb4145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:12.371005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc42b90-52dc-47ad-883a-6d0388eb4145 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:12.395029 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1315/2660] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:14:12 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:14:12.396678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ef144-1bde-43cb-9a70-16ffa010d87e admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:12.407294 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ef144-1bde-43cb-9a70-16ffa010d87e admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:12.412032 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178d852-c275-4652-8e9f-74e8a5261d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:12.412725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178d852-c275-4652-8e9f-74e8a5261d17 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:12.413003 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178d852-c275-4652-8e9f-74e8a5261d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:12.413281 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178d852-c275-4652-8e9f-74e8a5261d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:12.414033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178d852-c275-4652-8e9f-74e8a5261d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:12.415617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178d852-c275-4652-8e9f-74e8a5261d17 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:12.426257 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1346/2661] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:14:12.430166 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc42b90-52dc-47ad-883a-6d0388eb4145 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:12.440689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc42b90-52dc-47ad-883a-6d0388eb4145 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:12.445690 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee64ca3-4206-4d10-b43e-7cafca4dd99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:12.446349 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee64ca3-4206-4d10-b43e-7cafca4dd99c admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:12.446618 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee64ca3-4206-4d10-b43e-7cafca4dd99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:12.446889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee64ca3-4206-4d10-b43e-7cafca4dd99c admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:12.447630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee64ca3-4206-4d10-b43e-7cafca4dd99c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:12.449445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee64ca3-4206-4d10-b43e-7cafca4dd99c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:12.459680 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1316/2662] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:14:12.463836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178d852-c275-4652-8e9f-74e8a5261d17 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:12.473943 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178d852-c275-4652-8e9f-74e8a5261d17 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:12.478759 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c1079-68b5-48bb-bf2a-3e1994cd67e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:12.479641 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c1079-68b5-48bb-bf2a-3e1994cd67e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:12.479920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c1079-68b5-48bb-bf2a-3e1994cd67e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:12.480236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c1079-68b5-48bb-bf2a-3e1994cd67e2 admin admin] PATH_INFO: `/v3/projects/c944841d55a642b0b458d9d11160bdcb/users/b0c4534f652d4226948382da57151f09/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:12.494953 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c1079-68b5-48bb-bf2a-3e1994cd67e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c944841d55a642b0b458d9d11160bdcb, user_id=b0c4534f652d4226948382da57151f09, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:12.498042 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c1079-68b5-48bb-bf2a-3e1994cd67e2 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:12.512752 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1347/2663] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:14:12 2022] PUT /identity/v3/projects/c944841d55a642b0b458d9d11160bdcb/users/b0c4534f652d4226948382da57151f09/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:14:12.523372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0450159-7833-4a51-b81d-5eec23f85cc4 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:12.523662 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0450159-7833-4a51-b81d-5eec23f85cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:12.523936 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0450159-7833-4a51-b81d-5eec23f85cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:12.552775 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a0450159-7833-4a51-b81d-5eec23f85cc4 None None] MFA Rules not processed for user `b0c4534f652d4226948382da57151f09`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:12.553987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a0450159-7833-4a51-b81d-5eec23f85cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:12.621325 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1317/2664] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:14:12.626003 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c1079-68b5-48bb-bf2a-3e1994cd67e2 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:12.693758 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c1079-68b5-48bb-bf2a-3e1994cd67e2 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:12.698951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c89d9-2c3d-4e56-b118-2e9c8433d54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:12.699608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c89d9-2c3d-4e56-b118-2e9c8433d54f admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:12.699886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c89d9-2c3d-4e56-b118-2e9c8433d54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:12.700163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c89d9-2c3d-4e56-b118-2e9c8433d54f admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:12.701023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c89d9-2c3d-4e56-b118-2e9c8433d54f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:12.702924 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c89d9-2c3d-4e56-b118-2e9c8433d54f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:12.728175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:12.737289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1348/2665] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:14:12 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:14:12.741458 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0450159-7833-4a51-b81d-5eec23f85cc4 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:12.752479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0450159-7833-4a51-b81d-5eec23f85cc4 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:12.812784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cad83-3eb2-4e1d-8586-36f2d337be7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:12.813488 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cad83-3eb2-4e1d-8586-36f2d337be7f admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:12.813771 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cad83-3eb2-4e1d-8586-36f2d337be7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:12.814046 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cad83-3eb2-4e1d-8586-36f2d337be7f admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:12.814739 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cad83-3eb2-4e1d-8586-36f2d337be7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:12.816731 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cad83-3eb2-4e1d-8586-36f2d337be7f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:12.838661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1318/2666] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:14:12 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:14:12.842778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c89d9-2c3d-4e56-b118-2e9c8433d54f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:12.853851 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c89d9-2c3d-4e56-b118-2e9c8433d54f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:12.859282 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36bd56-c8b5-409c-a37b-4e6c85592e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:12.859981 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36bd56-c8b5-409c-a37b-4e6c85592e5c admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:12.860289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36bd56-c8b5-409c-a37b-4e6c85592e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:12.860598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36bd56-c8b5-409c-a37b-4e6c85592e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:12.861323 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36bd56-c8b5-409c-a37b-4e6c85592e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:12.862919 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36bd56-c8b5-409c-a37b-4e6c85592e5c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:12.875352 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1349/2667] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:14:12.880866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cad83-3eb2-4e1d-8586-36f2d337be7f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:12.886414 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:12.892513 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cad83-3eb2-4e1d-8586-36f2d337be7f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:12.898918 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab520ab-4d94-4ac9-b0f1-99e64b2ea651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:12.899866 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab520ab-4d94-4ac9-b0f1-99e64b2ea651 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:12.900149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab520ab-4d94-4ac9-b0f1-99e64b2ea651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:12.900437 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab520ab-4d94-4ac9-b0f1-99e64b2ea651 admin admin] PATH_INFO: `/v3/projects/17bfcd519be34ee19a0d026369962cba/users/40952b4834614538afea3ca41b88cab8/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:12.916528 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab520ab-4d94-4ac9-b0f1-99e64b2ea651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17bfcd519be34ee19a0d026369962cba, user_id=40952b4834614538afea3ca41b88cab8, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:12.919277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab520ab-4d94-4ac9-b0f1-99e64b2ea651 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:12.934477 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1319/2668] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:14:12 2022] PUT /identity/v3/projects/17bfcd519be34ee19a0d026369962cba/users/40952b4834614538afea3ca41b88cab8/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:14:12.938444 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36bd56-c8b5-409c-a37b-4e6c85592e5c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:12.949064 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36bd56-c8b5-409c-a37b-4e6c85592e5c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:13.008620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236e0aa-cf86-4128-9b6d-7d474a651701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:13.009322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236e0aa-cf86-4128-9b6d-7d474a651701 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:13.009565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236e0aa-cf86-4128-9b6d-7d474a651701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:13.009836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236e0aa-cf86-4128-9b6d-7d474a651701 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:13.010560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236e0aa-cf86-4128-9b6d-7d474a651701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:13.012390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236e0aa-cf86-4128-9b6d-7d474a651701 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:13.023276 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1350/2669] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:14:13.027512 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab520ab-4d94-4ac9-b0f1-99e64b2ea651 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:13.038272 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab520ab-4d94-4ac9-b0f1-99e64b2ea651 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:13.043790 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595a74a-f8c5-466b-8220-a8ad5b8e3e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:13.044701 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595a74a-f8c5-466b-8220-a8ad5b8e3e16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:13.044979 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595a74a-f8c5-466b-8220-a8ad5b8e3e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:13.045248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595a74a-f8c5-466b-8220-a8ad5b8e3e16 admin admin] PATH_INFO: `/v3/projects/17bfcd519be34ee19a0d026369962cba/users/40952b4834614538afea3ca41b88cab8/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:13.047320 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595a74a-f8c5-466b-8220-a8ad5b8e3e16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17bfcd519be34ee19a0d026369962cba, user_id=40952b4834614538afea3ca41b88cab8, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:13.050076 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595a74a-f8c5-466b-8220-a8ad5b8e3e16 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:13.061383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1320/2670] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:14:13 2022] PUT /identity/v3/projects/17bfcd519be34ee19a0d026369962cba/users/40952b4834614538afea3ca41b88cab8/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:13.073086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ef18b-c1e9-4a8f-8114-81c5c84b25bd None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:13.073425 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ef18b-c1e9-4a8f-8114-81c5c84b25bd None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:13.073702 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ef18b-c1e9-4a8f-8114-81c5c84b25bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:13.101912 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-645ef18b-c1e9-4a8f-8114-81c5c84b25bd None None] MFA Rules not processed for user `40952b4834614538afea3ca41b88cab8`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:13.103166 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-645ef18b-c1e9-4a8f-8114-81c5c84b25bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:13.175948 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1351/2671] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:13.188055 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595a74a-f8c5-466b-8220-a8ad5b8e3e16 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:13.191012 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a595a74a-f8c5-466b-8220-a8ad5b8e3e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:13.266414 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595a74a-f8c5-466b-8220-a8ad5b8e3e16 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:13.271378 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4ae99-096f-4eaf-add6-d375a20f9b8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:13.272027 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4ae99-096f-4eaf-add6-d375a20f9b8a service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:13.272311 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4ae99-096f-4eaf-add6-d375a20f9b8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:13.272620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4ae99-096f-4eaf-add6-d375a20f9b8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:13.332200 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4ae99-096f-4eaf-add6-d375a20f9b8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:13.333691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e4ae99-096f-4eaf-add6-d375a20f9b8a service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:13.343557 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1321/2672] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:14:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:14:15.896220 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ef18b-c1e9-4a8f-8114-81c5c84b25bd None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:15.959889 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ef18b-c1e9-4a8f-8114-81c5c84b25bd None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:15.965190 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f63c84-20b6-44f6-9434-abf3572ebb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:15.966088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f63c84-20b6-44f6-9434-abf3572ebb1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:15.966364 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f63c84-20b6-44f6-9434-abf3572ebb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:15.966635 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f63c84-20b6-44f6-9434-abf3572ebb1a admin admin] PATH_INFO: `/v3/users/b0c4534f652d4226948382da57151f09` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:15.967804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f63c84-20b6-44f6-9434-abf3572ebb1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0c4534f652d4226948382da57151f09)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:15.970076 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f63c84-20b6-44f6-9434-abf3572ebb1a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:15.983931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:16.023932 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-62f63c84-20b6-44f6-9434-abf3572ebb1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0c4534f652d4226948382da57151f09'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:16.031361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-62f63c84-20b6-44f6-9434-abf3572ebb1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0c4534f652d4226948382da57151f09'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:16.032115 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-62f63c84-20b6-44f6-9434-abf3572ebb1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0c4534f652d4226948382da57151f09'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:16.039160 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-62f63c84-20b6-44f6-9434-abf3572ebb1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0c4534f652d4226948382da57151f09'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:16.039812 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-62f63c84-20b6-44f6-9434-abf3572ebb1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0c4534f652d4226948382da57151f09'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:16.060832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1352/2673] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:14:15 2022] DELETE /identity/v3/users/b0c4534f652d4226948382da57151f09 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:16.065217 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db737377-c624-4190-a558-2b22aed9f6d9 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:16.065728 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db737377-c624-4190-a558-2b22aed9f6d9 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:16.065728 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db737377-c624-4190-a558-2b22aed9f6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:16.083923 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-db737377-c624-4190-a558-2b22aed9f6d9 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:16.085266 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-db737377-c624-4190-a558-2b22aed9f6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:16.148809 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1322/2674] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:14:16.159736 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f63c84-20b6-44f6-9434-abf3572ebb1a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:16.161431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-62f63c84-20b6-44f6-9434-abf3572ebb1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:16.223752 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f63c84-20b6-44f6-9434-abf3572ebb1a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:16.227530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a51f6-6c40-4e34-91d8-0c452c2ba574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:16.228188 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a51f6-6c40-4e34-91d8-0c452c2ba574 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:16.228496 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a51f6-6c40-4e34-91d8-0c452c2ba574 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:16.228777 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a51f6-6c40-4e34-91d8-0c452c2ba574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:16.239353 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a51f6-6c40-4e34-91d8-0c452c2ba574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:16.240917 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a51f6-6c40-4e34-91d8-0c452c2ba574 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:16.251270 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1353/2675] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:14:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:14:16.540789 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db737377-c624-4190-a558-2b22aed9f6d9 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:16.542348 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-db737377-c624-4190-a558-2b22aed9f6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:16.554824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db737377-c624-4190-a558-2b22aed9f6d9 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:16.560326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166725e1-a399-4b9f-b52a-5698916ff197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:16.561668 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166725e1-a399-4b9f-b52a-5698916ff197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:16.561912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166725e1-a399-4b9f-b52a-5698916ff197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:16.562223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166725e1-a399-4b9f-b52a-5698916ff197 admin admin] PATH_INFO: `/v3/projects/c944841d55a642b0b458d9d11160bdcb` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:16.563610 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166725e1-a399-4b9f-b52a-5698916ff197 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c944841d55a642b0b458d9d11160bdcb)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:16.565542 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166725e1-a399-4b9f-b52a-5698916ff197 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:16.583583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:16.620874 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-166725e1-a399-4b9f-b52a-5698916ff197 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c944841d55a642b0b458d9d11160bdcb', 'request_id': 'req-166725e1-a399-4b9f-b52a-5698916ff197'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:16.627048 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-166725e1-a399-4b9f-b52a-5698916ff197 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c944841d55a642b0b458d9d11160bdcb', 'request_id': 'req-166725e1-a399-4b9f-b52a-5698916ff197'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:16.634520 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-166725e1-a399-4b9f-b52a-5698916ff197 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c944841d55a642b0b458d9d11160bdcb', 'request_id': 'req-166725e1-a399-4b9f-b52a-5698916ff197'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:16.647573 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-166725e1-a399-4b9f-b52a-5698916ff197 admin admin] The token cache is being invalidate because project c944841d55a642b0b458d9d11160bdcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:14:16.647859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-166725e1-a399-4b9f-b52a-5698916ff197 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:16.648507 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-166725e1-a399-4b9f-b52a-5698916ff197 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:16.650122 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1323/2676] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:14:16 2022] DELETE /identity/v3/projects/c944841d55a642b0b458d9d11160bdcb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:16.654129 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a51f6-6c40-4e34-91d8-0c452c2ba574 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:16.663874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a51f6-6c40-4e34-91d8-0c452c2ba574 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:16.728986 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862efb81-8624-4ed6-9ffd-91854fc09e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:16.729864 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862efb81-8624-4ed6-9ffd-91854fc09e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:16.730200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862efb81-8624-4ed6-9ffd-91854fc09e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:16.730477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862efb81-8624-4ed6-9ffd-91854fc09e67 admin admin] PATH_INFO: `/v3/users/40952b4834614538afea3ca41b88cab8` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:16.731861 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862efb81-8624-4ed6-9ffd-91854fc09e67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40952b4834614538afea3ca41b88cab8)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:16.733808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862efb81-8624-4ed6-9ffd-91854fc09e67 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:16.775796 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:16.789112 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-862efb81-8624-4ed6-9ffd-91854fc09e67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40952b4834614538afea3ca41b88cab8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:16.796573 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-862efb81-8624-4ed6-9ffd-91854fc09e67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40952b4834614538afea3ca41b88cab8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:16.797256 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-862efb81-8624-4ed6-9ffd-91854fc09e67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40952b4834614538afea3ca41b88cab8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:16.805135 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-862efb81-8624-4ed6-9ffd-91854fc09e67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40952b4834614538afea3ca41b88cab8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:16.805797 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-862efb81-8624-4ed6-9ffd-91854fc09e67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40952b4834614538afea3ca41b88cab8'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:16.823402 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1354/2677] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:14:16 2022] DELETE /identity/v3/users/40952b4834614538afea3ca41b88cab8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:17.108005 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166725e1-a399-4b9f-b52a-5698916ff197 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:17.109698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-166725e1-a399-4b9f-b52a-5698916ff197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:17.135397 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166725e1-a399-4b9f-b52a-5698916ff197 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:17.192835 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5b96d-b7a9-4fdf-a9f0-79aaa32a4e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:17.193706 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5b96d-b7a9-4fdf-a9f0-79aaa32a4e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:17.193986 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5b96d-b7a9-4fdf-a9f0-79aaa32a4e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:17.194258 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5b96d-b7a9-4fdf-a9f0-79aaa32a4e03 admin admin] PATH_INFO: `/v3/projects/17bfcd519be34ee19a0d026369962cba` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:17.195511 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d5b96d-b7a9-4fdf-a9f0-79aaa32a4e03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17bfcd519be34ee19a0d026369962cba)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:17.197448 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d5b96d-b7a9-4fdf-a9f0-79aaa32a4e03 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:17.253503 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-67d5b96d-b7a9-4fdf-a9f0-79aaa32a4e03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17bfcd519be34ee19a0d026369962cba', 'request_id': 'req-67d5b96d-b7a9-4fdf-a9f0-79aaa32a4e03'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:17.259763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-67d5b96d-b7a9-4fdf-a9f0-79aaa32a4e03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17bfcd519be34ee19a0d026369962cba', 'request_id': 'req-67d5b96d-b7a9-4fdf-a9f0-79aaa32a4e03'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:17.267103 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-67d5b96d-b7a9-4fdf-a9f0-79aaa32a4e03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17bfcd519be34ee19a0d026369962cba', 'request_id': 'req-67d5b96d-b7a9-4fdf-a9f0-79aaa32a4e03'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:17.281561 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-67d5b96d-b7a9-4fdf-a9f0-79aaa32a4e03 admin admin] The token cache is being invalidate because project 17bfcd519be34ee19a0d026369962cba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:14:17.281841 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-67d5b96d-b7a9-4fdf-a9f0-79aaa32a4e03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:17.282504 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-67d5b96d-b7a9-4fdf-a9f0-79aaa32a4e03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:17.284051 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1324/2678] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:14:17 2022] DELETE /identity/v3/projects/17bfcd519be34ee19a0d026369962cba => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:17.289259 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1799fa-dafa-4ed0-ae7f-8452608da251 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:17.289722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1799fa-dafa-4ed0-ae7f-8452608da251 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:17.289830 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1799fa-dafa-4ed0-ae7f-8452608da251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:17.309696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1a1799fa-dafa-4ed0-ae7f-8452608da251 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:17.310923 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1a1799fa-dafa-4ed0-ae7f-8452608da251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:17.372841 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1355/2679] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:17.384634 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a900be4-b70c-4a0a-a4ab-b2b2b6d0ccd0 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:17.384921 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a900be4-b70c-4a0a-a4ab-b2b2b6d0ccd0 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:17.385192 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a900be4-b70c-4a0a-a4ab-b2b2b6d0ccd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:17.404229 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-7a900be4-b70c-4a0a-a4ab-b2b2b6d0ccd0 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:17.405448 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7a900be4-b70c-4a0a-a4ab-b2b2b6d0ccd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:17.413933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1325/2680] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:17.418667 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1799fa-dafa-4ed0-ae7f-8452608da251 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:17.429303 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1799fa-dafa-4ed0-ae7f-8452608da251 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:17.435450 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec025843-14f1-47e8-97a7-68fa40333387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:17.436124 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec025843-14f1-47e8-97a7-68fa40333387 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:17.436377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec025843-14f1-47e8-97a7-68fa40333387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:17.436700 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec025843-14f1-47e8-97a7-68fa40333387 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:17.437456 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec025843-14f1-47e8-97a7-68fa40333387 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:14:17.437757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec025843-14f1-47e8-97a7-68fa40333387 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:17.439390 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec025843-14f1-47e8-97a7-68fa40333387 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:17.452501 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1356/2681] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:14:17.457671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a900be4-b70c-4a0a-a4ab-b2b2b6d0ccd0 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:17.473063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a900be4-b70c-4a0a-a4ab-b2b2b6d0ccd0 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:17.481893 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50430e45-de83-4d01-b6e3-93f48e15ea41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:17.482552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50430e45-de83-4d01-b6e3-93f48e15ea41 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:17.482834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50430e45-de83-4d01-b6e3-93f48e15ea41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:17.483105 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50430e45-de83-4d01-b6e3-93f48e15ea41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:17.483800 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50430e45-de83-4d01-b6e3-93f48e15ea41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:17.485775 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50430e45-de83-4d01-b6e3-93f48e15ea41 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:17.515368 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1326/2682] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:14:17 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:14:17.519619 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec025843-14f1-47e8-97a7-68fa40333387 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:17.530591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec025843-14f1-47e8-97a7-68fa40333387 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:17.590151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f798eb-d0eb-4268-bc6d-54a032229d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:17.590823 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f798eb-d0eb-4268-bc6d-54a032229d01 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:17.591102 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f798eb-d0eb-4268-bc6d-54a032229d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:17.591373 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f798eb-d0eb-4268-bc6d-54a032229d01 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:17.592070 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f798eb-d0eb-4268-bc6d-54a032229d01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:17.594176 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f798eb-d0eb-4268-bc6d-54a032229d01 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:17.621633 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1357/2683] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:14:17 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:14:17.625696 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50430e45-de83-4d01-b6e3-93f48e15ea41 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:17.632483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:17.636811 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50430e45-de83-4d01-b6e3-93f48e15ea41 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:17.643687 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8f90b-ac34-412f-8f57-67a12b4c4155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:17.644364 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8f90b-ac34-412f-8f57-67a12b4c4155 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:17.644673 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8f90b-ac34-412f-8f57-67a12b4c4155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:17.644949 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8f90b-ac34-412f-8f57-67a12b4c4155 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:17.645867 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8f90b-ac34-412f-8f57-67a12b4c4155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:17.647796 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8f90b-ac34-412f-8f57-67a12b4c4155 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:17.658999 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1327/2684] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:14:17.663251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f798eb-d0eb-4268-bc6d-54a032229d01 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:17.673850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f798eb-d0eb-4268-bc6d-54a032229d01 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:17.679729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c28dc-503a-4553-8256-de60d36c8a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:17.680385 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c28dc-503a-4553-8256-de60d36c8a2e admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:17.680698 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c28dc-503a-4553-8256-de60d36c8a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:17.680969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c28dc-503a-4553-8256-de60d36c8a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:17.681693 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c28dc-503a-4553-8256-de60d36c8a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:17.683343 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c28dc-503a-4553-8256-de60d36c8a2e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:17.693756 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1358/2685] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:14:17.698248 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8f90b-ac34-412f-8f57-67a12b4c4155 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:17.708767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8f90b-ac34-412f-8f57-67a12b4c4155 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:17.714648 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd6f65-9519-4b4f-b8e6-a0e19a757469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:17.715528 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd6f65-9519-4b4f-b8e6-a0e19a757469 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:17.715803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd6f65-9519-4b4f-b8e6-a0e19a757469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:17.716073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd6f65-9519-4b4f-b8e6-a0e19a757469 admin admin] PATH_INFO: `/v3/projects/95a3f6d6fdba415ea67a50764333bbc3/users/47a43a5920a64c37a8ff6ee63c005995/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:17.732613 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd6f65-9519-4b4f-b8e6-a0e19a757469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95a3f6d6fdba415ea67a50764333bbc3, user_id=47a43a5920a64c37a8ff6ee63c005995, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:17.735241 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd6f65-9519-4b4f-b8e6-a0e19a757469 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:17.749889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1328/2686] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:14:17 2022] PUT /identity/v3/projects/95a3f6d6fdba415ea67a50764333bbc3/users/47a43a5920a64c37a8ff6ee63c005995/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:17.766579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37de1b-da56-4ac2-95ac-92151749e0ac None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:17.766926 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37de1b-da56-4ac2-95ac-92151749e0ac None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:17.767295 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37de1b-da56-4ac2-95ac-92151749e0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:17.797508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-3f37de1b-da56-4ac2-95ac-92151749e0ac None None] MFA Rules not processed for user `47a43a5920a64c37a8ff6ee63c005995`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:17.798731 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3f37de1b-da56-4ac2-95ac-92151749e0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:17.804230 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:17.867123 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1359/2687] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:17.872645 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd6f65-9519-4b4f-b8e6-a0e19a757469 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:17.941801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd6f65-9519-4b4f-b8e6-a0e19a757469 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:17.946532 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ecae6-5a4a-4d46-8841-e8b13a1d9d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:17.947185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ecae6-5a4a-4d46-8841-e8b13a1d9d33 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:17.947466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ecae6-5a4a-4d46-8841-e8b13a1d9d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:17.947741 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ecae6-5a4a-4d46-8841-e8b13a1d9d33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:17.948481 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ecae6-5a4a-4d46-8841-e8b13a1d9d33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:17.950492 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ecae6-5a4a-4d46-8841-e8b13a1d9d33 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:17.979327 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1329/2688] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:14:17 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:14:17.983717 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37de1b-da56-4ac2-95ac-92151749e0ac None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:17.996472 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37de1b-da56-4ac2-95ac-92151749e0ac None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:18.056085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ded133-46ee-48df-bde4-c5abc15f1f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:18.056807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ded133-46ee-48df-bde4-c5abc15f1f99 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:18.057090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ded133-46ee-48df-bde4-c5abc15f1f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:18.057350 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ded133-46ee-48df-bde4-c5abc15f1f99 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:18.058042 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ded133-46ee-48df-bde4-c5abc15f1f99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:18.060677 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ded133-46ee-48df-bde4-c5abc15f1f99 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:18.085988 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1360/2689] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:14:17 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:14:18.090670 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ecae6-5a4a-4d46-8841-e8b13a1d9d33 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:18.101751 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ecae6-5a4a-4d46-8841-e8b13a1d9d33 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:18.107440 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206981d-0e3a-4451-9a9f-a99b0decbb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:18.108117 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206981d-0e3a-4451-9a9f-a99b0decbb8f admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:18.108752 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206981d-0e3a-4451-9a9f-a99b0decbb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:18.108752 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206981d-0e3a-4451-9a9f-a99b0decbb8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:18.109416 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206981d-0e3a-4451-9a9f-a99b0decbb8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:18.111035 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206981d-0e3a-4451-9a9f-a99b0decbb8f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:18.122022 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1330/2690] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:14:18.126130 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ded133-46ee-48df-bde4-c5abc15f1f99 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:18.136894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ded133-46ee-48df-bde4-c5abc15f1f99 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:18.143145 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175a8db-0671-436c-9703-8a452b8681f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:18.143804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175a8db-0671-436c-9703-8a452b8681f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:18.144079 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175a8db-0671-436c-9703-8a452b8681f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:18.144350 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175a8db-0671-436c-9703-8a452b8681f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:18.145093 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175a8db-0671-436c-9703-8a452b8681f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:18.146680 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175a8db-0671-436c-9703-8a452b8681f1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:18.157386 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1361/2691] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:14:18.161703 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206981d-0e3a-4451-9a9f-a99b0decbb8f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:18.172130 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206981d-0e3a-4451-9a9f-a99b0decbb8f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:18.177375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91344f1e-3374-4845-8ff4-ff59ccff8443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:18.178631 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91344f1e-3374-4845-8ff4-ff59ccff8443 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:18.178912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91344f1e-3374-4845-8ff4-ff59ccff8443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:18.179186 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91344f1e-3374-4845-8ff4-ff59ccff8443 admin admin] PATH_INFO: `/v3/projects/80e84fec55f54dcc9b060d899a76d462/users/66ea5fb732564e08a9175a4bc0098639/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:18.194517 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91344f1e-3374-4845-8ff4-ff59ccff8443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e84fec55f54dcc9b060d899a76d462, user_id=66ea5fb732564e08a9175a4bc0098639, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:18.197206 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91344f1e-3374-4845-8ff4-ff59ccff8443 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:18.211978 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1331/2692] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:14:18 2022] PUT /identity/v3/projects/80e84fec55f54dcc9b060d899a76d462/users/66ea5fb732564e08a9175a4bc0098639/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:18.223741 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590e823-ce3f-4a3e-93df-df48ac24b9a5 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:18.224029 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590e823-ce3f-4a3e-93df-df48ac24b9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:18.224306 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590e823-ce3f-4a3e-93df-df48ac24b9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:18.253031 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8590e823-ce3f-4a3e-93df-df48ac24b9a5 None None] MFA Rules not processed for user `66ea5fb732564e08a9175a4bc0098639`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:18.254152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8590e823-ce3f-4a3e-93df-df48ac24b9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:18.340447 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1362/2693] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:18.345446 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91344f1e-3374-4845-8ff4-ff59ccff8443 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:18.419542 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91344f1e-3374-4845-8ff4-ff59ccff8443 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:18.424687 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f76edf-8c46-4116-a68d-b6ed433cdf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:18.425340 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f76edf-8c46-4116-a68d-b6ed433cdf78 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:18.425620 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f76edf-8c46-4116-a68d-b6ed433cdf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:18.425898 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f76edf-8c46-4116-a68d-b6ed433cdf78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:18.426618 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f76edf-8c46-4116-a68d-b6ed433cdf78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:18.428554 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f76edf-8c46-4116-a68d-b6ed433cdf78 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:18.457092 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1332/2694] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:14:18 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:14:18.461185 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590e823-ce3f-4a3e-93df-df48ac24b9a5 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:18.472029 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590e823-ce3f-4a3e-93df-df48ac24b9a5 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:18.483595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:18.536066 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3046208-8496-484d-bebf-60848c6a6293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:18.536679 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3046208-8496-484d-bebf-60848c6a6293 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:18.536967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3046208-8496-484d-bebf-60848c6a6293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:18.537246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3046208-8496-484d-bebf-60848c6a6293 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:18.538014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3046208-8496-484d-bebf-60848c6a6293 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:18.540043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3046208-8496-484d-bebf-60848c6a6293 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:18.563587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1363/2695] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:14:18 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:14:18.564957 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f76edf-8c46-4116-a68d-b6ed433cdf78 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:18.571726 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:18.575968 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f76edf-8c46-4116-a68d-b6ed433cdf78 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:18.582518 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5f477-6c37-4ebe-b52b-716a80f9ea3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:18.583180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5f477-6c37-4ebe-b52b-716a80f9ea3a admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:18.583459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5f477-6c37-4ebe-b52b-716a80f9ea3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:18.583733 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5f477-6c37-4ebe-b52b-716a80f9ea3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:18.584491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5f477-6c37-4ebe-b52b-716a80f9ea3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:18.586095 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5f477-6c37-4ebe-b52b-716a80f9ea3a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:18.597042 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1333/2696] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:14:18.601380 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3046208-8496-484d-bebf-60848c6a6293 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:18.612061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3046208-8496-484d-bebf-60848c6a6293 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:18.618668 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caabdf90-29fd-4019-b6d1-18bad011630a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:18.619595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabdf90-29fd-4019-b6d1-18bad011630a admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:18.619876 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabdf90-29fd-4019-b6d1-18bad011630a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:18.620151 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabdf90-29fd-4019-b6d1-18bad011630a admin admin] PATH_INFO: `/v3/projects/7116df2578fa42ca9edfa6784ead9308/users/d5b2e5eee9574eb185e0217762b9f3c4/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:18.636003 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caabdf90-29fd-4019-b6d1-18bad011630a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7116df2578fa42ca9edfa6784ead9308, user_id=d5b2e5eee9574eb185e0217762b9f3c4, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:18.638726 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caabdf90-29fd-4019-b6d1-18bad011630a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:18.656023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1364/2697] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:14:18 2022] PUT /identity/v3/projects/7116df2578fa42ca9edfa6784ead9308/users/d5b2e5eee9574eb185e0217762b9f3c4/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:18.660607 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5f477-6c37-4ebe-b52b-716a80f9ea3a admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:18.676018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5f477-6c37-4ebe-b52b-716a80f9ea3a admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:18.737888 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02375f6d-2661-464f-962b-9ba7017bbf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:18.738561 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02375f6d-2661-464f-962b-9ba7017bbf32 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:18.738842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02375f6d-2661-464f-962b-9ba7017bbf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:18.739119 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02375f6d-2661-464f-962b-9ba7017bbf32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:18.739882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02375f6d-2661-464f-962b-9ba7017bbf32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:18.741591 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02375f6d-2661-464f-962b-9ba7017bbf32 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:18.751318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1334/2698] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:14:18.755575 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caabdf90-29fd-4019-b6d1-18bad011630a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:18.766367 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caabdf90-29fd-4019-b6d1-18bad011630a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:18.771656 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cc53d-6eea-4e2f-a574-3f1f1a659ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:18.773203 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cc53d-6eea-4e2f-a574-3f1f1a659ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:18.773687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cc53d-6eea-4e2f-a574-3f1f1a659ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:18.774164 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3cc53d-6eea-4e2f-a574-3f1f1a659ac1 admin admin] PATH_INFO: `/v3/projects/7116df2578fa42ca9edfa6784ead9308/users/d5b2e5eee9574eb185e0217762b9f3c4/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:18.777069 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3cc53d-6eea-4e2f-a574-3f1f1a659ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7116df2578fa42ca9edfa6784ead9308, user_id=d5b2e5eee9574eb185e0217762b9f3c4, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:18.780054 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3cc53d-6eea-4e2f-a574-3f1f1a659ac1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:18.792050 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1365/2699] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:14:18 2022] PUT /identity/v3/projects/7116df2578fa42ca9edfa6784ead9308/users/d5b2e5eee9574eb185e0217762b9f3c4/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:18.806076 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd23ad-28c6-4447-994c-e574828d1c63 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:18.806364 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd23ad-28c6-4447-994c-e574828d1c63 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:18.806639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd23ad-28c6-4447-994c-e574828d1c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:18.837147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-bdbd23ad-28c6-4447-994c-e574828d1c63 None None] MFA Rules not processed for user `d5b2e5eee9574eb185e0217762b9f3c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:18.838380 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bdbd23ad-28c6-4447-994c-e574828d1c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:18.916575 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1335/2700] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:18.929159 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cc53d-6eea-4e2f-a574-3f1f1a659ac1 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:18.931147 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0d3cc53d-6eea-4e2f-a574-3f1f1a659ac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:18.990218 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3cc53d-6eea-4e2f-a574-3f1f1a659ac1 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:18.993967 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efaf72-a5ce-4c11-bdfb-697594e9b028 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:18.994609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efaf72-a5ce-4c11-bdfb-697594e9b028 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:18.994891 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efaf72-a5ce-4c11-bdfb-697594e9b028 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:18.995169 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efaf72-a5ce-4c11-bdfb-697594e9b028 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:19.053643 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36efaf72-a5ce-4c11-bdfb-697594e9b028 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:19.055007 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36efaf72-a5ce-4c11-bdfb-697594e9b028 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:19.064925 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1366/2701] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:14:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:14:20.316905 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd23ad-28c6-4447-994c-e574828d1c63 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:20.318633 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-bdbd23ad-28c6-4447-994c-e574828d1c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:20.390100 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd23ad-28c6-4447-994c-e574828d1c63 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:20.395304 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c517ebf-93d8-4ca0-af1d-2244c4be2d62 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474c468a610140f8a5b18d5a1ed1d18e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:20.395953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c517ebf-93d8-4ca0-af1d-2244c4be2d62 service placement] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:20.396231 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c517ebf-93d8-4ca0-af1d-2244c4be2d62 service placement] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:20.396538 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c517ebf-93d8-4ca0-af1d-2244c4be2d62 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:20.398431 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5c517ebf-93d8-4ca0-af1d-2244c4be2d62 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:20.410851 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c517ebf-93d8-4ca0-af1d-2244c4be2d62 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:20.412617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c517ebf-93d8-4ca0-af1d-2244c4be2d62 service placement] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:20.424098 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1336/2702] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:14:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:14:20.586206 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efaf72-a5ce-4c11-bdfb-697594e9b028 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:20.587639 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-36efaf72-a5ce-4c11-bdfb-697594e9b028 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:20.599318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efaf72-a5ce-4c11-bdfb-697594e9b028 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:20.603595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a8097-ecaa-4fd3-be90-b853f6c87ee8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:20.604323 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a8097-ecaa-4fd3-be90-b853f6c87ee8 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:20.604696 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a8097-ecaa-4fd3-be90-b853f6c87ee8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:20.605067 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a8097-ecaa-4fd3-be90-b853f6c87ee8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:20.625971 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:20.667700 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a8097-ecaa-4fd3-be90-b853f6c87ee8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:20.669085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a8097-ecaa-4fd3-be90-b853f6c87ee8 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:20.678171 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1367/2703] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:14:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:14:21.248596 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c517ebf-93d8-4ca0-af1d-2244c4be2d62 service placement] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:21.254514 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=100 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:21.259522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c517ebf-93d8-4ca0-af1d-2244c4be2d62 service placement] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:21.263495 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84208a2b-464b-46b1-8cfd-8777487d3b87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:21.264503 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84208a2b-464b-46b1-8cfd-8777487d3b87 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:21.264782 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84208a2b-464b-46b1-8cfd-8777487d3b87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:21.265051 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84208a2b-464b-46b1-8cfd-8777487d3b87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:21.274383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84208a2b-464b-46b1-8cfd-8777487d3b87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:21.275820 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84208a2b-464b-46b1-8cfd-8777487d3b87 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:21.285478 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1337/2704] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:14:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:14:25.236841 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a8097-ecaa-4fd3-be90-b853f6c87ee8 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:25.310360 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a8097-ecaa-4fd3-be90-b853f6c87ee8 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:25.315892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a582e50-6d14-4e05-abab-70f94eae1a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:25.316867 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a582e50-6d14-4e05-abab-70f94eae1a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:25.317213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a582e50-6d14-4e05-abab-70f94eae1a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:25.317489 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a582e50-6d14-4e05-abab-70f94eae1a86 admin admin] PATH_INFO: `/v3/users/47a43a5920a64c37a8ff6ee63c005995` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:25.318753 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a582e50-6d14-4e05-abab-70f94eae1a86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47a43a5920a64c37a8ff6ee63c005995)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:25.320718 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a582e50-6d14-4e05-abab-70f94eae1a86 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:25.371641 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3a582e50-6d14-4e05-abab-70f94eae1a86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47a43a5920a64c37a8ff6ee63c005995'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:25.378388 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3a582e50-6d14-4e05-abab-70f94eae1a86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47a43a5920a64c37a8ff6ee63c005995'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:25.379047 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3a582e50-6d14-4e05-abab-70f94eae1a86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47a43a5920a64c37a8ff6ee63c005995'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:25.387987 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3a582e50-6d14-4e05-abab-70f94eae1a86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47a43a5920a64c37a8ff6ee63c005995'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:25.389320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-3a582e50-6d14-4e05-abab-70f94eae1a86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47a43a5920a64c37a8ff6ee63c005995'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:25.412808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1368/2705] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:14:25 2022] DELETE /identity/v3/users/47a43a5920a64c37a8ff6ee63c005995 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:25.418493 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61614c8d-f357-44a8-9f86-8253b787a7ce None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:25.418978 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61614c8d-f357-44a8-9f86-8253b787a7ce None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:25.419073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61614c8d-f357-44a8-9f86-8253b787a7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:25.442329 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-61614c8d-f357-44a8-9f86-8253b787a7ce None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:25.443542 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-61614c8d-f357-44a8-9f86-8253b787a7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:25.510993 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1338/2706] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:25.523473 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a582e50-6d14-4e05-abab-70f94eae1a86 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:25.525173 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3a582e50-6d14-4e05-abab-70f94eae1a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:25.574875 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:25.591718 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a582e50-6d14-4e05-abab-70f94eae1a86 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:25.596527 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ebbb3-4343-477d-8dfc-c06b794ec3b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:25.597179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ebbb3-4343-477d-8dfc-c06b794ec3b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:25.597457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ebbb3-4343-477d-8dfc-c06b794ec3b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:25.597732 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ebbb3-4343-477d-8dfc-c06b794ec3b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:25.607480 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ebbb3-4343-477d-8dfc-c06b794ec3b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:25.608947 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ebbb3-4343-477d-8dfc-c06b794ec3b7 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:25.618351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1369/2707] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:14:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:14:25.909403 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61614c8d-f357-44a8-9f86-8253b787a7ce None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:25.910924 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-61614c8d-f357-44a8-9f86-8253b787a7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:25.927139 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61614c8d-f357-44a8-9f86-8253b787a7ce None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:25.932941 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08a617-9294-4ff5-8257-3b634979175c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:25.933952 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08a617-9294-4ff5-8257-3b634979175c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:25.934223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08a617-9294-4ff5-8257-3b634979175c admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:25.934647 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08a617-9294-4ff5-8257-3b634979175c admin admin] PATH_INFO: `/v3/projects/95a3f6d6fdba415ea67a50764333bbc3` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:25.935804 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08a617-9294-4ff5-8257-3b634979175c admin admin] RBAC: Authorizing `identity:delete_project(project_id=95a3f6d6fdba415ea67a50764333bbc3)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:25.937826 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08a617-9294-4ff5-8257-3b634979175c admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:25.986092 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3a08a617-9294-4ff5-8257-3b634979175c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95a3f6d6fdba415ea67a50764333bbc3', 'request_id': 'req-3a08a617-9294-4ff5-8257-3b634979175c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:25.992919 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3a08a617-9294-4ff5-8257-3b634979175c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95a3f6d6fdba415ea67a50764333bbc3', 'request_id': 'req-3a08a617-9294-4ff5-8257-3b634979175c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:25.999296 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3a08a617-9294-4ff5-8257-3b634979175c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95a3f6d6fdba415ea67a50764333bbc3', 'request_id': 'req-3a08a617-9294-4ff5-8257-3b634979175c'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:26.018835 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3a08a617-9294-4ff5-8257-3b634979175c admin admin] The token cache is being invalidate because project 95a3f6d6fdba415ea67a50764333bbc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:14:26.019221 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3a08a617-9294-4ff5-8257-3b634979175c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:26.020330 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-3a08a617-9294-4ff5-8257-3b634979175c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:26.022859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1339/2708] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:14:25 2022] DELETE /identity/v3/projects/95a3f6d6fdba415ea67a50764333bbc3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:26.028894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ebbb3-4343-477d-8dfc-c06b794ec3b7 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:26.039513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ebbb3-4343-477d-8dfc-c06b794ec3b7 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:26.112791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f45ae0-409d-4bd0-a7e9-5f1bd6360146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:26.113706 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f45ae0-409d-4bd0-a7e9-5f1bd6360146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:26.113989 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f45ae0-409d-4bd0-a7e9-5f1bd6360146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:26.114265 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f45ae0-409d-4bd0-a7e9-5f1bd6360146 admin admin] PATH_INFO: `/v3/users/66ea5fb732564e08a9175a4bc0098639` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:26.115528 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f45ae0-409d-4bd0-a7e9-5f1bd6360146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66ea5fb732564e08a9175a4bc0098639)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:26.117511 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f45ae0-409d-4bd0-a7e9-5f1bd6360146 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:26.182598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e8f45ae0-409d-4bd0-a7e9-5f1bd6360146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66ea5fb732564e08a9175a4bc0098639'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:26.190125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e8f45ae0-409d-4bd0-a7e9-5f1bd6360146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66ea5fb732564e08a9175a4bc0098639'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:26.190840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e8f45ae0-409d-4bd0-a7e9-5f1bd6360146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66ea5fb732564e08a9175a4bc0098639'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:26.199801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e8f45ae0-409d-4bd0-a7e9-5f1bd6360146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66ea5fb732564e08a9175a4bc0098639'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:26.200447 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-e8f45ae0-409d-4bd0-a7e9-5f1bd6360146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66ea5fb732564e08a9175a4bc0098639'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:26.218393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1370/2709] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:14:26 2022] DELETE /identity/v3/users/66ea5fb732564e08a9175a4bc0098639 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:26.502888 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08a617-9294-4ff5-8257-3b634979175c admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:26.504345 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-3a08a617-9294-4ff5-8257-3b634979175c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:26.518224 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08a617-9294-4ff5-8257-3b634979175c admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:26.582914 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be963f63-cb1c-4fda-b9cc-605597e854bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:26.583788 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be963f63-cb1c-4fda-b9cc-605597e854bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:26.584091 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be963f63-cb1c-4fda-b9cc-605597e854bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:26.584364 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be963f63-cb1c-4fda-b9cc-605597e854bd admin admin] PATH_INFO: `/v3/projects/80e84fec55f54dcc9b060d899a76d462` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:26.585616 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be963f63-cb1c-4fda-b9cc-605597e854bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=80e84fec55f54dcc9b060d899a76d462)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:26.587512 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be963f63-cb1c-4fda-b9cc-605597e854bd admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:26.604344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:26.637325 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-be963f63-cb1c-4fda-b9cc-605597e854bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80e84fec55f54dcc9b060d899a76d462', 'request_id': 'req-be963f63-cb1c-4fda-b9cc-605597e854bd'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:26.644075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-be963f63-cb1c-4fda-b9cc-605597e854bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80e84fec55f54dcc9b060d899a76d462', 'request_id': 'req-be963f63-cb1c-4fda-b9cc-605597e854bd'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:26.650625 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-be963f63-cb1c-4fda-b9cc-605597e854bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80e84fec55f54dcc9b060d899a76d462', 'request_id': 'req-be963f63-cb1c-4fda-b9cc-605597e854bd'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:26.665923 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-be963f63-cb1c-4fda-b9cc-605597e854bd admin admin] The token cache is being invalidate because project 80e84fec55f54dcc9b060d899a76d462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:14:26.666210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-be963f63-cb1c-4fda-b9cc-605597e854bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:26.666911 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-be963f63-cb1c-4fda-b9cc-605597e854bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:26.668527 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1340/2710] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:14:26 2022] DELETE /identity/v3/projects/80e84fec55f54dcc9b060d899a76d462 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:26.672671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f45ae0-409d-4bd0-a7e9-5f1bd6360146 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:26.674120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-e8f45ae0-409d-4bd0-a7e9-5f1bd6360146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:26.687365 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f45ae0-409d-4bd0-a7e9-5f1bd6360146 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:26.750112 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37552f-7c46-40c2-9079-bb056e1769d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:26.750990 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37552f-7c46-40c2-9079-bb056e1769d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:26.751267 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37552f-7c46-40c2-9079-bb056e1769d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:26.751539 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37552f-7c46-40c2-9079-bb056e1769d2 admin admin] PATH_INFO: `/v3/users/d5b2e5eee9574eb185e0217762b9f3c4` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:26.752932 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37552f-7c46-40c2-9079-bb056e1769d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5b2e5eee9574eb185e0217762b9f3c4)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:26.754903 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37552f-7c46-40c2-9079-bb056e1769d2 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:26.757311 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:26.806617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-0e37552f-7c46-40c2-9079-bb056e1769d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5b2e5eee9574eb185e0217762b9f3c4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:26.814179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-0e37552f-7c46-40c2-9079-bb056e1769d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5b2e5eee9574eb185e0217762b9f3c4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:26.814931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-0e37552f-7c46-40c2-9079-bb056e1769d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5b2e5eee9574eb185e0217762b9f3c4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:26.824533 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-0e37552f-7c46-40c2-9079-bb056e1769d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5b2e5eee9574eb185e0217762b9f3c4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:26.825185 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-0e37552f-7c46-40c2-9079-bb056e1769d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5b2e5eee9574eb185e0217762b9f3c4'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:26.843407 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1371/2711] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:14:26 2022] DELETE /identity/v3/users/d5b2e5eee9574eb185e0217762b9f3c4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:27.127077 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be963f63-cb1c-4fda-b9cc-605597e854bd admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:27.128566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-be963f63-cb1c-4fda-b9cc-605597e854bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:27.142538 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be963f63-cb1c-4fda-b9cc-605597e854bd admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:27.204071 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c7139-d26b-44ce-96e0-b4eb880c38b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:27.204978 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c7139-d26b-44ce-96e0-b4eb880c38b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:27.205254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c7139-d26b-44ce-96e0-b4eb880c38b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:27.205531 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c7139-d26b-44ce-96e0-b4eb880c38b8 admin admin] PATH_INFO: `/v3/projects/7116df2578fa42ca9edfa6784ead9308` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:27.206786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c7139-d26b-44ce-96e0-b4eb880c38b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7116df2578fa42ca9edfa6784ead9308)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:27.208762 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c7139-d26b-44ce-96e0-b4eb880c38b8 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:27.257366 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d29c7139-d26b-44ce-96e0-b4eb880c38b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7116df2578fa42ca9edfa6784ead9308', 'request_id': 'req-d29c7139-d26b-44ce-96e0-b4eb880c38b8'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:27.262918 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d29c7139-d26b-44ce-96e0-b4eb880c38b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7116df2578fa42ca9edfa6784ead9308', 'request_id': 'req-d29c7139-d26b-44ce-96e0-b4eb880c38b8'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:27.269317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d29c7139-d26b-44ce-96e0-b4eb880c38b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7116df2578fa42ca9edfa6784ead9308', 'request_id': 'req-d29c7139-d26b-44ce-96e0-b4eb880c38b8'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:27.282839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d29c7139-d26b-44ce-96e0-b4eb880c38b8 admin admin] The token cache is being invalidate because project 7116df2578fa42ca9edfa6784ead9308 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:14:27.283123 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d29c7139-d26b-44ce-96e0-b4eb880c38b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:27.283739 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-d29c7139-d26b-44ce-96e0-b4eb880c38b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:27.285323 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1341/2712] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:14:27 2022] DELETE /identity/v3/projects/7116df2578fa42ca9edfa6784ead9308 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:27.290243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be387e9c-38f6-4ab8-9eee-18abbdad2cb1 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:27.290554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be387e9c-38f6-4ab8-9eee-18abbdad2cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:27.290857 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be387e9c-38f6-4ab8-9eee-18abbdad2cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:27.309225 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-be387e9c-38f6-4ab8-9eee-18abbdad2cb1 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:27.310459 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-be387e9c-38f6-4ab8-9eee-18abbdad2cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:27.375046 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1372/2713] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:27.383049 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26963a78-f946-4bfd-bc98-eaf9e9835042 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:27.383352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26963a78-f946-4bfd-bc98-eaf9e9835042 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:27.383614 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26963a78-f946-4bfd-bc98-eaf9e9835042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:27.403114 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-26963a78-f946-4bfd-bc98-eaf9e9835042 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:27.404327 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-26963a78-f946-4bfd-bc98-eaf9e9835042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:27.413790 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1342/2714] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:27.418244 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be387e9c-38f6-4ab8-9eee-18abbdad2cb1 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:27.428713 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be387e9c-38f6-4ab8-9eee-18abbdad2cb1 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:27.434564 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3267cb-d963-483d-96d7-84103c6c9573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:27.435215 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3267cb-d963-483d-96d7-84103c6c9573 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:27.435490 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3267cb-d963-483d-96d7-84103c6c9573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:27.435778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3267cb-d963-483d-96d7-84103c6c9573 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:27.436536 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3267cb-d963-483d-96d7-84103c6c9573 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:14:27.436840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3267cb-d963-483d-96d7-84103c6c9573 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:27.438427 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3267cb-d963-483d-96d7-84103c6c9573 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:27.450530 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1373/2715] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:14:27.455104 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26963a78-f946-4bfd-bc98-eaf9e9835042 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:27.465045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26963a78-f946-4bfd-bc98-eaf9e9835042 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:27.470661 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424711d7-4f57-4880-84a3-499f16d55992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:27.471286 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424711d7-4f57-4880-84a3-499f16d55992 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:27.471568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424711d7-4f57-4880-84a3-499f16d55992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:27.471843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424711d7-4f57-4880-84a3-499f16d55992 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:27.472568 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424711d7-4f57-4880-84a3-499f16d55992 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:27.474522 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424711d7-4f57-4880-84a3-499f16d55992 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:27.505491 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1343/2716] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:14:27 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:14:27.509656 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3267cb-d963-483d-96d7-84103c6c9573 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:27.520710 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3267cb-d963-483d-96d7-84103c6c9573 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:27.578311 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313eed2-d4a7-4e44-bcad-adad6b755c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:27.579006 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313eed2-d4a7-4e44-bcad-adad6b755c83 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:27.579285 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313eed2-d4a7-4e44-bcad-adad6b755c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:27.579559 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313eed2-d4a7-4e44-bcad-adad6b755c83 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:27.580255 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313eed2-d4a7-4e44-bcad-adad6b755c83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:27.582269 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313eed2-d4a7-4e44-bcad-adad6b755c83 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:27.598323 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:27.605230 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1374/2717] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:14:27 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:14:27.609466 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424711d7-4f57-4880-84a3-499f16d55992 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:27.620537 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424711d7-4f57-4880-84a3-499f16d55992 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:27.625343 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809eea7-11dc-4a1c-b00b-901158bf8024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:27.625939 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809eea7-11dc-4a1c-b00b-901158bf8024 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:27.626213 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809eea7-11dc-4a1c-b00b-901158bf8024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:27.626484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809eea7-11dc-4a1c-b00b-901158bf8024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:27.627214 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809eea7-11dc-4a1c-b00b-901158bf8024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:27.628916 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809eea7-11dc-4a1c-b00b-901158bf8024 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:27.630218 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:27.645180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1344/2718] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:14:27.649178 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313eed2-d4a7-4e44-bcad-adad6b755c83 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:27.662249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313eed2-d4a7-4e44-bcad-adad6b755c83 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:27.667255 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d2c64-c0af-4611-bdd2-b2664bd7fbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:27.667984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d2c64-c0af-4611-bdd2-b2664bd7fbaa admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:27.668311 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d2c64-c0af-4611-bdd2-b2664bd7fbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:27.668628 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d2c64-c0af-4611-bdd2-b2664bd7fbaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:27.669358 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d2c64-c0af-4611-bdd2-b2664bd7fbaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:27.670951 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d2c64-c0af-4611-bdd2-b2664bd7fbaa admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:27.682157 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1375/2719] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:14:27.688907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809eea7-11dc-4a1c-b00b-901158bf8024 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:27.699115 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809eea7-11dc-4a1c-b00b-901158bf8024 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:27.704281 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4b128-6bc7-424e-b1fa-3469b2c35f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:27.705197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4b128-6bc7-424e-b1fa-3469b2c35f0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:27.705503 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4b128-6bc7-424e-b1fa-3469b2c35f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:27.705778 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4b128-6bc7-424e-b1fa-3469b2c35f0d admin admin] PATH_INFO: `/v3/projects/300775afba3141ec943347bf19288903/users/75f259900ce3402f89f02f46fefa1158/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:27.722631 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec4b128-6bc7-424e-b1fa-3469b2c35f0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=300775afba3141ec943347bf19288903, user_id=75f259900ce3402f89f02f46fefa1158, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:27.726224 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec4b128-6bc7-424e-b1fa-3469b2c35f0d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:27.742187 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1345/2720] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:14:27 2022] PUT /identity/v3/projects/300775afba3141ec943347bf19288903/users/75f259900ce3402f89f02f46fefa1158/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:27.812038 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265381bc-053d-4eb0-b0a3-f13409f5a024 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:27.812359 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265381bc-053d-4eb0-b0a3-f13409f5a024 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:27.812657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265381bc-053d-4eb0-b0a3-f13409f5a024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:27.842424 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-265381bc-053d-4eb0-b0a3-f13409f5a024 None None] MFA Rules not processed for user `75f259900ce3402f89f02f46fefa1158`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:27.843752 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-265381bc-053d-4eb0-b0a3-f13409f5a024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:27.916322 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1376/2721] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:27 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:27.928259 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4b128-6bc7-424e-b1fa-3469b2c35f0d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:27.930284 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-cec4b128-6bc7-424e-b1fa-3469b2c35f0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:27.987531 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4b128-6bc7-424e-b1fa-3469b2c35f0d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:27.991689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c245c27-dd6f-451f-934e-98ab0b95623e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cba086728824ec68fb0ee90fb3f99dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:27.992324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c245c27-dd6f-451f-934e-98ab0b95623e service glance] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:27.992628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c245c27-dd6f-451f-934e-98ab0b95623e service glance] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:27.992898 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c245c27-dd6f-451f-934e-98ab0b95623e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:28.007236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c245c27-dd6f-451f-934e-98ab0b95623e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:28.009191 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c245c27-dd6f-451f-934e-98ab0b95623e service glance] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:28.019746 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1346/2722] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:14:27 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:14:35.658160 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748c721-8955-4217-b5de-b11f9ff36ec4 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:35.658929 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748c721-8955-4217-b5de-b11f9ff36ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:35.658929 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748c721-8955-4217-b5de-b11f9ff36ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:35.676691 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-0748c721-8955-4217-b5de-b11f9ff36ec4 None None] MFA Rules not processed for user `5c2141524dc546259cd993ace59e33f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:35.677916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0748c721-8955-4217-b5de-b11f9ff36ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:35.737986 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1377/2723] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:36.049750 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c245c27-dd6f-451f-934e-98ab0b95623e service glance] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:36.051318 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4c245c27-dd6f-451f-934e-98ab0b95623e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:36.121923 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c245c27-dd6f-451f-934e-98ab0b95623e service glance] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:36.125895 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b77ad2-775a-450c-9ecd-3d247c613881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:36.126597 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b77ad2-775a-450c-9ecd-3d247c613881 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:36.126882 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b77ad2-775a-450c-9ecd-3d247c613881 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:36.127149 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b77ad2-775a-450c-9ecd-3d247c613881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:36.129082 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b3b77ad2-775a-450c-9ecd-3d247c613881 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:36.139483 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b77ad2-775a-450c-9ecd-3d247c613881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:36.140951 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b77ad2-775a-450c-9ecd-3d247c613881 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:36.150659 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1347/2724] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:14:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:14:45.731367 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=63 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:46.145879 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=64,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:53.785442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748c721-8955-4217-b5de-b11f9ff36ec4 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:53.856540 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748c721-8955-4217-b5de-b11f9ff36ec4 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:53.861517 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b60ca5-7172-49c2-9eab-c22b7b3b31d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:53.862408 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b60ca5-7172-49c2-9eab-c22b7b3b31d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:53.862750 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b60ca5-7172-49c2-9eab-c22b7b3b31d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:53.862987 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b60ca5-7172-49c2-9eab-c22b7b3b31d1 admin admin] PATH_INFO: `/v3/users/75f259900ce3402f89f02f46fefa1158` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:53.864200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b60ca5-7172-49c2-9eab-c22b7b3b31d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75f259900ce3402f89f02f46fefa1158)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:53.866192 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b60ca5-7172-49c2-9eab-c22b7b3b31d1 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:53.916198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f2b60ca5-7172-49c2-9eab-c22b7b3b31d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75f259900ce3402f89f02f46fefa1158'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:53.922942 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f2b60ca5-7172-49c2-9eab-c22b7b3b31d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75f259900ce3402f89f02f46fefa1158'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:53.923624 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f2b60ca5-7172-49c2-9eab-c22b7b3b31d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75f259900ce3402f89f02f46fefa1158'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:53.932000 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f2b60ca5-7172-49c2-9eab-c22b7b3b31d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75f259900ce3402f89f02f46fefa1158'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:53.932826 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-f2b60ca5-7172-49c2-9eab-c22b7b3b31d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75f259900ce3402f89f02f46fefa1158'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:53.949583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1378/2725] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:14:53 2022] DELETE /identity/v3/users/75f259900ce3402f89f02f46fefa1158 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:53.954126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e50694-4e5b-434a-a05f-63b48a9118ce None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:53.954634 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e50694-4e5b-434a-a05f-63b48a9118ce None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:53.954738 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e50694-4e5b-434a-a05f-63b48a9118ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:53.972227 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-d6e50694-4e5b-434a-a05f-63b48a9118ce None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:53.973541 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d6e50694-4e5b-434a-a05f-63b48a9118ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:54.034786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1348/2726] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:54.045932 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b60ca5-7172-49c2-9eab-c22b7b3b31d1 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:54.047598 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-f2b60ca5-7172-49c2-9eab-c22b7b3b31d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:54.106702 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b60ca5-7172-49c2-9eab-c22b7b3b31d1 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:54.111237 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c8525-96ad-4e78-b06c-32b3397ffaf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:54.111884 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c8525-96ad-4e78-b06c-32b3397ffaf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:54.112162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c8525-96ad-4e78-b06c-32b3397ffaf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:54.112440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c8525-96ad-4e78-b06c-32b3397ffaf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:54.122724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c8525-96ad-4e78-b06c-32b3397ffaf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:54.124153 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c8525-96ad-4e78-b06c-32b3397ffaf4 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:54.133832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1379/2727] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:14:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:14:54.403439 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e50694-4e5b-434a-a05f-63b48a9118ce None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:54.404960 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-d6e50694-4e5b-434a-a05f-63b48a9118ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:54.417262 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e50694-4e5b-434a-a05f-63b48a9118ce None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:54.423037 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f7daf-86d1-49fd-9a1e-c44ead42fa72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:54.423444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f7daf-86d1-49fd-9a1e-c44ead42fa72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:54.423720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f7daf-86d1-49fd-9a1e-c44ead42fa72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:54.423987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f7daf-86d1-49fd-9a1e-c44ead42fa72 admin admin] PATH_INFO: `/v3/projects/300775afba3141ec943347bf19288903` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:54.425231 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f7daf-86d1-49fd-9a1e-c44ead42fa72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=300775afba3141ec943347bf19288903)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:54.427132 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f7daf-86d1-49fd-9a1e-c44ead42fa72 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:54.473891 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8e3f7daf-86d1-49fd-9a1e-c44ead42fa72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '300775afba3141ec943347bf19288903', 'request_id': 'req-8e3f7daf-86d1-49fd-9a1e-c44ead42fa72'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:54.479617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8e3f7daf-86d1-49fd-9a1e-c44ead42fa72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '300775afba3141ec943347bf19288903', 'request_id': 'req-8e3f7daf-86d1-49fd-9a1e-c44ead42fa72'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:54.485823 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8e3f7daf-86d1-49fd-9a1e-c44ead42fa72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '300775afba3141ec943347bf19288903', 'request_id': 'req-8e3f7daf-86d1-49fd-9a1e-c44ead42fa72'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:54.499205 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8e3f7daf-86d1-49fd-9a1e-c44ead42fa72 admin admin] The token cache is being invalidate because project 300775afba3141ec943347bf19288903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:14:54.499487 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8e3f7daf-86d1-49fd-9a1e-c44ead42fa72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:54.500098 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-8e3f7daf-86d1-49fd-9a1e-c44ead42fa72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:14:54.501674 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1349/2728] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:14:54 2022] DELETE /identity/v3/projects/300775afba3141ec943347bf19288903 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:54.506757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853516f-7dee-4222-bc76-27ef4d7befaa None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:54.507061 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853516f-7dee-4222-bc76-27ef4d7befaa None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:54.507366 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853516f-7dee-4222-bc76-27ef4d7befaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:54.526152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5853516f-7dee-4222-bc76-27ef4d7befaa None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:54.527410 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5853516f-7dee-4222-bc76-27ef4d7befaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:54.545305 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:54.590312 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1380/2729] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:54.598099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2be23-add5-4b97-8f8a-f0c285f8959a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:54.598387 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2be23-add5-4b97-8f8a-f0c285f8959a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:54.598672 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2be23-add5-4b97-8f8a-f0c285f8959a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:54.617021 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-1be2be23-add5-4b97-8f8a-f0c285f8959a None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:54.618213 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1be2be23-add5-4b97-8f8a-f0c285f8959a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:54.631446 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1350/2730] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:54.638427 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853516f-7dee-4222-bc76-27ef4d7befaa None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:54.653630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853516f-7dee-4222-bc76-27ef4d7befaa None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:54.659086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2618bdc-443a-4464-800a-46567b3c8c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:54.659749 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2618bdc-443a-4464-800a-46567b3c8c53 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:54.660035 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2618bdc-443a-4464-800a-46567b3c8c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:54.660318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2618bdc-443a-4464-800a-46567b3c8c53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:54.661082 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2618bdc-443a-4464-800a-46567b3c8c53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:14:54.661392 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2618bdc-443a-4464-800a-46567b3c8c53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:54.662991 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2618bdc-443a-4464-800a-46567b3c8c53 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:54.674871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1381/2731] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:14:54.679657 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2be23-add5-4b97-8f8a-f0c285f8959a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:54.690864 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2be23-add5-4b97-8f8a-f0c285f8959a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:54.696507 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbbe49-661d-473b-b11d-5edd7a134811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:54.697164 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbbe49-661d-473b-b11d-5edd7a134811 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:54.697442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbbe49-661d-473b-b11d-5edd7a134811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:54.697714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbbe49-661d-473b-b11d-5edd7a134811 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:54.698750 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbbe49-661d-473b-b11d-5edd7a134811 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:54.700683 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbbe49-661d-473b-b11d-5edd7a134811 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:54.727571 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1351/2732] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:14:54 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:14:54.732008 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2618bdc-443a-4464-800a-46567b3c8c53 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:54.742377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2618bdc-443a-4464-800a-46567b3c8c53 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:54.815572 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77c78b-85c4-420f-91df-bae933609efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:54.816260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77c78b-85c4-420f-91df-bae933609efc admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:54.816597 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77c78b-85c4-420f-91df-bae933609efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:54.816875 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77c78b-85c4-420f-91df-bae933609efc admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:54.817579 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77c78b-85c4-420f-91df-bae933609efc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:54.819503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77c78b-85c4-420f-91df-bae933609efc admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:54.842750 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1382/2733] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:14:54 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:14:54.846978 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbbe49-661d-473b-b11d-5edd7a134811 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:54.858286 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbbe49-661d-473b-b11d-5edd7a134811 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:54.864045 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff387bc-0b2a-4ba3-b582-9b153ed4d987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:54.864769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff387bc-0b2a-4ba3-b582-9b153ed4d987 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:54.865050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff387bc-0b2a-4ba3-b582-9b153ed4d987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:54.865321 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff387bc-0b2a-4ba3-b582-9b153ed4d987 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:54.866081 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff387bc-0b2a-4ba3-b582-9b153ed4d987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:54.867696 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff387bc-0b2a-4ba3-b582-9b153ed4d987 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:54.879107 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1352/2734] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:14:54.883513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77c78b-85c4-420f-91df-bae933609efc admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:54.894008 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77c78b-85c4-420f-91df-bae933609efc admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:54.898963 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f647d-eff3-4083-be13-dbfb7bbdfc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:54.899624 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f647d-eff3-4083-be13-dbfb7bbdfc1e admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:54.899907 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f647d-eff3-4083-be13-dbfb7bbdfc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:54.900184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f647d-eff3-4083-be13-dbfb7bbdfc1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:54.900987 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f647d-eff3-4083-be13-dbfb7bbdfc1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:54.902619 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f647d-eff3-4083-be13-dbfb7bbdfc1e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:54.912745 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1383/2735] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:14:54.917010 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff387bc-0b2a-4ba3-b582-9b153ed4d987 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:54.926731 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff387bc-0b2a-4ba3-b582-9b153ed4d987 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:54.931464 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b22b9-d2dd-43b5-aefb-c4427f0811e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:54.932609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b22b9-d2dd-43b5-aefb-c4427f0811e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:54.932735 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b22b9-d2dd-43b5-aefb-c4427f0811e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:54.933018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b22b9-d2dd-43b5-aefb-c4427f0811e1 admin admin] PATH_INFO: `/v3/projects/5d13640c8dcd49008d2f16313fd6e7ee/users/03f1b99a0c3f42c08c5441b6311c450a/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:54.948299 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b22b9-d2dd-43b5-aefb-c4427f0811e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d13640c8dcd49008d2f16313fd6e7ee, user_id=03f1b99a0c3f42c08c5441b6311c450a, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:54.951305 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b22b9-d2dd-43b5-aefb-c4427f0811e1 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:54.972112 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1353/2736] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:14:54 2022] PUT /identity/v3/projects/5d13640c8dcd49008d2f16313fd6e7ee/users/03f1b99a0c3f42c08c5441b6311c450a/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:14:54.985920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4874b-4962-4a14-b951-6e860a900402 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:54.986205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4874b-4962-4a14-b951-6e860a900402 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:54.986482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4874b-4962-4a14-b951-6e860a900402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:55.016068 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a1b4874b-4962-4a14-b951-6e860a900402 None None] MFA Rules not processed for user `03f1b99a0c3f42c08c5441b6311c450a`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:55.017300 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a1b4874b-4962-4a14-b951-6e860a900402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:55.094056 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1384/2737] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:14:55.105435 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b22b9-d2dd-43b5-aefb-c4427f0811e1 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:14:55.107998 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5c1b22b9-d2dd-43b5-aefb-c4427f0811e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:55.131442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:14:55.167913 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b22b9-d2dd-43b5-aefb-c4427f0811e1 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:14:55.172142 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba3d7a-c751-4146-8d74-1d9685f54e71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:14:55.172811 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba3d7a-c751-4146-8d74-1d9685f54e71 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:55.173086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba3d7a-c751-4146-8d74-1d9685f54e71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:55.173357 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba3d7a-c751-4146-8d74-1d9685f54e71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:55.182929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba3d7a-c751-4146-8d74-1d9685f54e71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:14:55.184392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba3d7a-c751-4146-8d74-1d9685f54e71 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:14:55.193592 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1354/2738] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:14:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:14:56.549257 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56b05a-5c69-47a3-b415-a56f2b0b5b8c None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:14:56.549804 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56b05a-5c69-47a3-b415-a56f2b0b5b8c None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:14:56.549905 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56b05a-5c69-47a3-b415-a56f2b0b5b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:14:56.568515 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-7a56b05a-5c69-47a3-b415-a56f2b0b5b8c None None] MFA Rules not processed for user `03f1b99a0c3f42c08c5441b6311c450a`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:14:56.569715 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7a56b05a-5c69-47a3-b415-a56f2b0b5b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:14:56.578111 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1385/2739] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:14:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:15:05.189730 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=17 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:15:06.562847 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:15:09.241570 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a843780-2e84-4a7a-a4d0-0200f4efcc3a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:09.242079 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a843780-2e84-4a7a-a4d0-0200f4efcc3a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:09.242079 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a843780-2e84-4a7a-a4d0-0200f4efcc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:09.261479 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-1a843780-2e84-4a7a-a4d0-0200f4efcc3a None None] MFA Rules not processed for user `28b3e73439e64050b406a9f3e9b0160a`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:15:09.262704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1a843780-2e84-4a7a-a4d0-0200f4efcc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:09.323720 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1355/2740] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Jun 25 16:15:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:15:09.371836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56b05a-5c69-47a3-b415-a56f2b0b5b8c None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:09.373337 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7a56b05a-5c69-47a3-b415-a56f2b0b5b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:09.448620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56b05a-5c69-47a3-b415-a56f2b0b5b8c None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:09.454670 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54d776-a29b-498e-8806-ffba5dd7c13b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19788c4239ca44f9a31a7d32086c8947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:09.455321 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54d776-a29b-498e-8806-ffba5dd7c13b service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:09.455608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54d776-a29b-498e-8806-ffba5dd7c13b service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:09.455883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54d776-a29b-498e-8806-ffba5dd7c13b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:09.465287 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54d776-a29b-498e-8806-ffba5dd7c13b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:09.466891 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54d776-a29b-498e-8806-ffba5dd7c13b service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:09.475193 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1386/2741] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Jun 25 16:15:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 25 16:15:10.014078 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be1301-cd43-4fb1-ab56-c89ade59f74f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:10.014383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be1301-cd43-4fb1-ab56-c89ade59f74f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:10.014658 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be1301-cd43-4fb1-ab56-c89ade59f74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:10.032692 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-05be1301-cd43-4fb1-ab56-c89ade59f74f None None] MFA Rules not processed for user `28b3e73439e64050b406a9f3e9b0160a`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:15:10.033903 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-05be1301-cd43-4fb1-ab56-c89ade59f74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:10.041437 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1356/2742] 10.222.0.13 () {58 vars in 967 bytes} [Sat Jun 25 16:15:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:15:10.056457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54d776-a29b-498e-8806-ffba5dd7c13b service swift] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:10.067423 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54d776-a29b-498e-8806-ffba5dd7c13b service swift] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:10.073589 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f4ea4-d61f-4021-a2ab-ea59d98b2134 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19788c4239ca44f9a31a7d32086c8947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:10.074222 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f4ea4-d61f-4021-a2ab-ea59d98b2134 service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:10.074498 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f4ea4-d61f-4021-a2ab-ea59d98b2134 service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:10.074772 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f4ea4-d61f-4021-a2ab-ea59d98b2134 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:10.083249 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f4ea4-d61f-4021-a2ab-ea59d98b2134 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:10.084898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f4ea4-d61f-4021-a2ab-ea59d98b2134 service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:10.093008 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1387/2743] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Jun 25 16:15:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 25 16:15:10.289201 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062b5f1-b11e-466c-8937-6ce2d6c4e623 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:10.289695 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062b5f1-b11e-466c-8937-6ce2d6c4e623 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:10.289816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062b5f1-b11e-466c-8937-6ce2d6c4e623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:10.308142 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-6062b5f1-b11e-466c-8937-6ce2d6c4e623 None None] MFA Rules not processed for user `28b3e73439e64050b406a9f3e9b0160a`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:15:10.309407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-6062b5f1-b11e-466c-8937-6ce2d6c4e623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:10.317991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1357/2744] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Jun 25 16:15:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:15:10.330001 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f4ea4-d61f-4021-a2ab-ea59d98b2134 service swift] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:10.341534 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f4ea4-d61f-4021-a2ab-ea59d98b2134 service swift] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:10.348015 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28b37f-c69f-42a9-8e62-b0a5347f8cd6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19788c4239ca44f9a31a7d32086c8947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:10.348682 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28b37f-c69f-42a9-8e62-b0a5347f8cd6 service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:10.348960 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28b37f-c69f-42a9-8e62-b0a5347f8cd6 service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:10.349236 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28b37f-c69f-42a9-8e62-b0a5347f8cd6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:10.357979 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28b37f-c69f-42a9-8e62-b0a5347f8cd6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:10.359544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28b37f-c69f-42a9-8e62-b0a5347f8cd6 service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:10.367879 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1388/2745] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Jun 25 16:15:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 25 16:15:10.708282 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7a75b-3225-406b-b7e1-19b2913ad44a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:10.708604 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7a75b-3225-406b-b7e1-19b2913ad44a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:10.708863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e7a75b-3225-406b-b7e1-19b2913ad44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:10.729795 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-52e7a75b-3225-406b-b7e1-19b2913ad44a None None] MFA Rules not processed for user `28b3e73439e64050b406a9f3e9b0160a`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:15:10.730946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-52e7a75b-3225-406b-b7e1-19b2913ad44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:10.738717 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1358/2746] 10.222.0.13 () {58 vars in 967 bytes} [Sat Jun 25 16:15:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:15:10.751082 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28b37f-c69f-42a9-8e62-b0a5347f8cd6 service swift] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:10.762735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28b37f-c69f-42a9-8e62-b0a5347f8cd6 service swift] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:10.768693 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e99fa9-5dec-43cb-9a42-eec45a9abaee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19788c4239ca44f9a31a7d32086c8947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:10.769337 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e99fa9-5dec-43cb-9a42-eec45a9abaee service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:10.769620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e99fa9-5dec-43cb-9a42-eec45a9abaee service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:10.769894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e99fa9-5dec-43cb-9a42-eec45a9abaee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:10.778239 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e99fa9-5dec-43cb-9a42-eec45a9abaee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:10.779827 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e99fa9-5dec-43cb-9a42-eec45a9abaee service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:10.788036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1389/2747] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Jun 25 16:15:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 25 16:15:12.901806 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7a75b-3225-406b-b7e1-19b2913ad44a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:12.971913 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e7a75b-3225-406b-b7e1-19b2913ad44a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:12.976769 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64e422-0c6b-4e78-b682-7fd3d090f412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:12.977662 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64e422-0c6b-4e78-b682-7fd3d090f412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:12.977942 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64e422-0c6b-4e78-b682-7fd3d090f412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:12.978215 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64e422-0c6b-4e78-b682-7fd3d090f412 admin admin] PATH_INFO: `/v3/users/03f1b99a0c3f42c08c5441b6311c450a` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:12.979549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64e422-0c6b-4e78-b682-7fd3d090f412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03f1b99a0c3f42c08c5441b6311c450a)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:12.981489 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64e422-0c6b-4e78-b682-7fd3d090f412 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:13.030918 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ad64e422-0c6b-4e78-b682-7fd3d090f412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03f1b99a0c3f42c08c5441b6311c450a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:13.031565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ad64e422-0c6b-4e78-b682-7fd3d090f412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03f1b99a0c3f42c08c5441b6311c450a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:13.040300 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ad64e422-0c6b-4e78-b682-7fd3d090f412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03f1b99a0c3f42c08c5441b6311c450a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:13.048200 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ad64e422-0c6b-4e78-b682-7fd3d090f412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03f1b99a0c3f42c08c5441b6311c450a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:13.048920 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ad64e422-0c6b-4e78-b682-7fd3d090f412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03f1b99a0c3f42c08c5441b6311c450a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:13.067548 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1359/2748] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:15:12 2022] DELETE /identity/v3/users/03f1b99a0c3f42c08c5441b6311c450a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:15:13.072193 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd4188-e792-4d7a-9d77-ce0512665898 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:13.072846 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd4188-e792-4d7a-9d77-ce0512665898 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:13.072846 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd4188-e792-4d7a-9d77-ce0512665898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:13.093213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-bcfd4188-e792-4d7a-9d77-ce0512665898 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:15:13.094426 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-bcfd4188-e792-4d7a-9d77-ce0512665898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:13.157125 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1390/2749] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:15:13.166819 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64e422-0c6b-4e78-b682-7fd3d090f412 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:13.168387 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ad64e422-0c6b-4e78-b682-7fd3d090f412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:13.175519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:15:13.230822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64e422-0c6b-4e78-b682-7fd3d090f412 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:13.235339 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce2b9c-aed5-4df1-829f-2094f79d488e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:13.235988 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce2b9c-aed5-4df1-829f-2094f79d488e service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:13.236271 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce2b9c-aed5-4df1-829f-2094f79d488e service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:13.236575 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce2b9c-aed5-4df1-829f-2094f79d488e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:13.246508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce2b9c-aed5-4df1-829f-2094f79d488e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:13.247958 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce2b9c-aed5-4df1-829f-2094f79d488e service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:13.257469 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1360/2750] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:15:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:15:13.451003 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd4188-e792-4d7a-9d77-ce0512665898 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:13.452453 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-bcfd4188-e792-4d7a-9d77-ce0512665898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:13.465326 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd4188-e792-4d7a-9d77-ce0512665898 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:13.470070 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92719b39-16d7-49e0-a785-9815bdae3c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:13.471075 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92719b39-16d7-49e0-a785-9815bdae3c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:13.471353 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92719b39-16d7-49e0-a785-9815bdae3c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:13.471625 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92719b39-16d7-49e0-a785-9815bdae3c6d admin admin] PATH_INFO: `/v3/projects/5d13640c8dcd49008d2f16313fd6e7ee` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:13.473004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92719b39-16d7-49e0-a785-9815bdae3c6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d13640c8dcd49008d2f16313fd6e7ee)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:13.474836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92719b39-16d7-49e0-a785-9815bdae3c6d admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:13.522229 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:15:13.525527 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-92719b39-16d7-49e0-a785-9815bdae3c6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d13640c8dcd49008d2f16313fd6e7ee', 'request_id': 'req-92719b39-16d7-49e0-a785-9815bdae3c6d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:13.532041 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-92719b39-16d7-49e0-a785-9815bdae3c6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d13640c8dcd49008d2f16313fd6e7ee', 'request_id': 'req-92719b39-16d7-49e0-a785-9815bdae3c6d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:13.538383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-92719b39-16d7-49e0-a785-9815bdae3c6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d13640c8dcd49008d2f16313fd6e7ee', 'request_id': 'req-92719b39-16d7-49e0-a785-9815bdae3c6d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:13.552513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-92719b39-16d7-49e0-a785-9815bdae3c6d admin admin] The token cache is being invalidate because project 5d13640c8dcd49008d2f16313fd6e7ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:15:13.552811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-92719b39-16d7-49e0-a785-9815bdae3c6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:13.553421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-92719b39-16d7-49e0-a785-9815bdae3c6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:13.554956 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1391/2751] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:15:13 2022] DELETE /identity/v3/projects/5d13640c8dcd49008d2f16313fd6e7ee => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:15:23.253169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=24 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:15:23.549417 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:15:54.871392 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce2b9c-aed5-4df1-829f-2094f79d488e service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:54.872946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-0fce2b9c-aed5-4df1-829f-2094f79d488e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:54.890603 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce2b9c-aed5-4df1-829f-2094f79d488e service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:54.954608 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399027c1-1403-48f9-901a-e6fabd99a2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:54.955521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399027c1-1403-48f9-901a-e6fabd99a2f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:54.955773 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399027c1-1403-48f9-901a-e6fabd99a2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:54.956089 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399027c1-1403-48f9-901a-e6fabd99a2f4 admin admin] PATH_INFO: `/v3/users/5c2141524dc546259cd993ace59e33f3` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:54.957332 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399027c1-1403-48f9-901a-e6fabd99a2f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c2141524dc546259cd993ace59e33f3)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:54.959294 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399027c1-1403-48f9-901a-e6fabd99a2f4 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:55.011120 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-399027c1-1403-48f9-901a-e6fabd99a2f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c2141524dc546259cd993ace59e33f3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:55.012060 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-399027c1-1403-48f9-901a-e6fabd99a2f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c2141524dc546259cd993ace59e33f3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:55.028166 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-399027c1-1403-48f9-901a-e6fabd99a2f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c2141524dc546259cd993ace59e33f3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:55.036500 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-399027c1-1403-48f9-901a-e6fabd99a2f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c2141524dc546259cd993ace59e33f3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:55.037151 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-399027c1-1403-48f9-901a-e6fabd99a2f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c2141524dc546259cd993ace59e33f3'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:55.055659 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1361/2752] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:15:54 2022] DELETE /identity/v3/users/5c2141524dc546259cd993ace59e33f3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:15:55.255970 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92719b39-16d7-49e0-a785-9815bdae3c6d admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:55.257959 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-92719b39-16d7-49e0-a785-9815bdae3c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:55.286314 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92719b39-16d7-49e0-a785-9815bdae3c6d admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:55.347544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24bdf4-5a09-4215-a2e7-1ed92d497694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:55.348560 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24bdf4-5a09-4215-a2e7-1ed92d497694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:55.348914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24bdf4-5a09-4215-a2e7-1ed92d497694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:55.349294 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24bdf4-5a09-4215-a2e7-1ed92d497694 admin admin] PATH_INFO: `/v3/projects/93ca9301bea041ce881109c706904a0a` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:55.350591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24bdf4-5a09-4215-a2e7-1ed92d497694 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93ca9301bea041ce881109c706904a0a)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:55.352725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24bdf4-5a09-4215-a2e7-1ed92d497694 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:55.406803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9f24bdf4-5a09-4215-a2e7-1ed92d497694 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93ca9301bea041ce881109c706904a0a', 'request_id': 'req-9f24bdf4-5a09-4215-a2e7-1ed92d497694'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:55.413458 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9f24bdf4-5a09-4215-a2e7-1ed92d497694 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93ca9301bea041ce881109c706904a0a', 'request_id': 'req-9f24bdf4-5a09-4215-a2e7-1ed92d497694'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:55.419836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9f24bdf4-5a09-4215-a2e7-1ed92d497694 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93ca9301bea041ce881109c706904a0a', 'request_id': 'req-9f24bdf4-5a09-4215-a2e7-1ed92d497694'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:55.434821 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9f24bdf4-5a09-4215-a2e7-1ed92d497694 admin admin] The token cache is being invalidate because project 93ca9301bea041ce881109c706904a0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:15:55.435085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9f24bdf4-5a09-4215-a2e7-1ed92d497694 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:55.435786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-9f24bdf4-5a09-4215-a2e7-1ed92d497694 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:55.437358 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1392/2753] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:15:55 2022] DELETE /identity/v3/projects/93ca9301bea041ce881109c706904a0a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:15:55.442633 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb6a7c-efba-4c4c-b337-74810ee2aaaf None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:55.443022 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb6a7c-efba-4c4c-b337-74810ee2aaaf None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:55.443297 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb6a7c-efba-4c4c-b337-74810ee2aaaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:55.465735 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-fdcb6a7c-efba-4c4c-b337-74810ee2aaaf None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:15:55.467188 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-fdcb6a7c-efba-4c4c-b337-74810ee2aaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:55.532015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1362/2754] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:15:55.539813 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949cdd5-568e-482c-a15e-e43e559d0a74 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:55.540213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949cdd5-568e-482c-a15e-e43e559d0a74 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:55.540575 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949cdd5-568e-482c-a15e-e43e559d0a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:55.560159 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8949cdd5-568e-482c-a15e-e43e559d0a74 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:15:55.561414 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8949cdd5-568e-482c-a15e-e43e559d0a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:55.570081 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1393/2755] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:15:55.575246 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb6a7c-efba-4c4c-b337-74810ee2aaaf None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:55.591023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb6a7c-efba-4c4c-b337-74810ee2aaaf None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:55.597718 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae12f24-a842-499e-93f9-9154dfa98351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:55.598375 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae12f24-a842-499e-93f9-9154dfa98351 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:55.598651 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae12f24-a842-499e-93f9-9154dfa98351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:55.598916 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae12f24-a842-499e-93f9-9154dfa98351 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:55.599642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae12f24-a842-499e-93f9-9154dfa98351 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:15:55.599941 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae12f24-a842-499e-93f9-9154dfa98351 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:55.601572 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae12f24-a842-499e-93f9-9154dfa98351 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:55.613877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1363/2756] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:15:55.618455 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949cdd5-568e-482c-a15e-e43e559d0a74 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:55.628012 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949cdd5-568e-482c-a15e-e43e559d0a74 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:55.633757 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d9e8f-e923-4506-93fc-3f1b47c1bfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:55.634465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d9e8f-e923-4506-93fc-3f1b47c1bfa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:55.634740 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d9e8f-e923-4506-93fc-3f1b47c1bfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:55.635007 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d9e8f-e923-4506-93fc-3f1b47c1bfa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:55.635716 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d9e8f-e923-4506-93fc-3f1b47c1bfa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:55.637625 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d9e8f-e923-4506-93fc-3f1b47c1bfa2 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:55.666558 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1394/2757] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:15:55 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:15:55.671182 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae12f24-a842-499e-93f9-9154dfa98351 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:55.681697 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae12f24-a842-499e-93f9-9154dfa98351 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:55.685950 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:15:55.742696 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984e570-df9e-4f1c-95ac-eec20bbf64c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:55.743390 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984e570-df9e-4f1c-95ac-eec20bbf64c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:55.743676 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984e570-df9e-4f1c-95ac-eec20bbf64c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:55.743950 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984e570-df9e-4f1c-95ac-eec20bbf64c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:55.744685 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984e570-df9e-4f1c-95ac-eec20bbf64c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:55.746642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984e570-df9e-4f1c-95ac-eec20bbf64c0 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:55.773455 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1364/2758] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:15:55 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:15:55.774753 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d9e8f-e923-4506-93fc-3f1b47c1bfa2 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:55.785178 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d9e8f-e923-4506-93fc-3f1b47c1bfa2 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:55.790333 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d503b-f408-4581-be6f-230286611765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:55.791078 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d503b-f408-4581-be6f-230286611765 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:55.791361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d503b-f408-4581-be6f-230286611765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:55.791639 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d503b-f408-4581-be6f-230286611765 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:55.792382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d503b-f408-4581-be6f-230286611765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:55.794180 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d503b-f408-4581-be6f-230286611765 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:55.805316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1395/2759] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:15:55.809460 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984e570-df9e-4f1c-95ac-eec20bbf64c0 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:55.820349 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984e570-df9e-4f1c-95ac-eec20bbf64c0 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:55.825947 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573fb346-c965-405c-93d4-63cd9e000a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:55.826615 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fb346-c965-405c-93d4-63cd9e000a65 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:55.826896 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fb346-c965-405c-93d4-63cd9e000a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:55.827168 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573fb346-c965-405c-93d4-63cd9e000a65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:55.827905 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573fb346-c965-405c-93d4-63cd9e000a65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:55.829545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573fb346-c965-405c-93d4-63cd9e000a65 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:55.840059 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1365/2760] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:15:55.844288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d503b-f408-4581-be6f-230286611765 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:55.855315 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d503b-f408-4581-be6f-230286611765 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:55.860455 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4a838-6113-413a-a8d9-59859bee6e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:55.861396 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4a838-6113-413a-a8d9-59859bee6e76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:55.861679 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4a838-6113-413a-a8d9-59859bee6e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:55.861984 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4a838-6113-413a-a8d9-59859bee6e76 admin admin] PATH_INFO: `/v3/projects/734020a939754d21a86482a3d013cb6d/users/a1d4468465144f4cad71a8f5bfd3d697/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:55.877874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4a838-6113-413a-a8d9-59859bee6e76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=734020a939754d21a86482a3d013cb6d, user_id=a1d4468465144f4cad71a8f5bfd3d697, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:55.880559 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4a838-6113-413a-a8d9-59859bee6e76 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:55.899197 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1396/2761] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:15:55 2022] PUT /identity/v3/projects/734020a939754d21a86482a3d013cb6d/users/a1d4468465144f4cad71a8f5bfd3d697/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:15:55.909764 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7691e5-6d8f-43dd-92d5-fe90f90d7d5f None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:55.910268 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7691e5-6d8f-43dd-92d5-fe90f90d7d5f None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:55.910378 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7691e5-6d8f-43dd-92d5-fe90f90d7d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:55.937637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-dc7691e5-6d8f-43dd-92d5-fe90f90d7d5f None None] MFA Rules not processed for user `a1d4468465144f4cad71a8f5bfd3d697`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:15:55.938844 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-dc7691e5-6d8f-43dd-92d5-fe90f90d7d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:56.011063 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1366/2762] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:15:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:15:56.022749 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4a838-6113-413a-a8d9-59859bee6e76 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:56.024898 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-02d4a838-6113-413a-a8d9-59859bee6e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:56.057562 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:15:56.105325 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4a838-6113-413a-a8d9-59859bee6e76 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:56.111197 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716b67a-02d4-4aa0-b207-2d8fec22362d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:56.111871 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716b67a-02d4-4aa0-b207-2d8fec22362d service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:56.112187 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716b67a-02d4-4aa0-b207-2d8fec22362d service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:56.112442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716b67a-02d4-4aa0-b207-2d8fec22362d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:56.125482 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716b67a-02d4-4aa0-b207-2d8fec22362d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:56.127152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716b67a-02d4-4aa0-b207-2d8fec22362d service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:56.137640 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1397/2763] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:15:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:15:56.358500 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7691e5-6d8f-43dd-92d5-fe90f90d7d5f None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:56.423212 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7691e5-6d8f-43dd-92d5-fe90f90d7d5f None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:56.427987 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6269041-1ad3-47d5-87e3-f55211db6a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:56.428885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6269041-1ad3-47d5-87e3-f55211db6a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:56.429170 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6269041-1ad3-47d5-87e3-f55211db6a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:56.429442 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6269041-1ad3-47d5-87e3-f55211db6a99 admin admin] PATH_INFO: `/v3/users/a1d4468465144f4cad71a8f5bfd3d697` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:56.430803 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6269041-1ad3-47d5-87e3-f55211db6a99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1d4468465144f4cad71a8f5bfd3d697)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:56.432884 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6269041-1ad3-47d5-87e3-f55211db6a99 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:56.450054 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:15:56.488859 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b6269041-1ad3-47d5-87e3-f55211db6a99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1d4468465144f4cad71a8f5bfd3d697'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:56.489504 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b6269041-1ad3-47d5-87e3-f55211db6a99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1d4468465144f4cad71a8f5bfd3d697'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:56.498275 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b6269041-1ad3-47d5-87e3-f55211db6a99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1d4468465144f4cad71a8f5bfd3d697'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:56.505095 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b6269041-1ad3-47d5-87e3-f55211db6a99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1d4468465144f4cad71a8f5bfd3d697'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:56.505737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b6269041-1ad3-47d5-87e3-f55211db6a99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1d4468465144f4cad71a8f5bfd3d697'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:56.523064 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1367/2764] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:15:56 2022] DELETE /identity/v3/users/a1d4468465144f4cad71a8f5bfd3d697 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:15:56.527820 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b4ada-a7ea-4e1a-b371-d916bbc21574 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:56.528316 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b4ada-a7ea-4e1a-b371-d916bbc21574 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:56.528456 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b4ada-a7ea-4e1a-b371-d916bbc21574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:56.548224 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a16b4ada-a7ea-4e1a-b371-d916bbc21574 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:15:56.549494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a16b4ada-a7ea-4e1a-b371-d916bbc21574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:56.615778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1398/2765] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:15:56.625611 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6269041-1ad3-47d5-87e3-f55211db6a99 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:56.627268 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b6269041-1ad3-47d5-87e3-f55211db6a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:56.688281 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6269041-1ad3-47d5-87e3-f55211db6a99 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:56.692485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8884858-acb6-4a4a-8320-99fbd871b040 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:56.693141 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8884858-acb6-4a4a-8320-99fbd871b040 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:56.693420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8884858-acb6-4a4a-8320-99fbd871b040 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:56.693749 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8884858-acb6-4a4a-8320-99fbd871b040 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:56.703793 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8884858-acb6-4a4a-8320-99fbd871b040 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:56.705153 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8884858-acb6-4a4a-8320-99fbd871b040 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:56.715405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1368/2766] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:15:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:15:57.002841 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b4ada-a7ea-4e1a-b371-d916bbc21574 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:57.004308 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a16b4ada-a7ea-4e1a-b371-d916bbc21574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:57.023465 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b4ada-a7ea-4e1a-b371-d916bbc21574 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:57.029900 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cb818-47f7-451d-b0e9-43c6fd3cde14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:57.030707 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cb818-47f7-451d-b0e9-43c6fd3cde14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:57.030954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cb818-47f7-451d-b0e9-43c6fd3cde14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:57.031260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cb818-47f7-451d-b0e9-43c6fd3cde14 admin admin] PATH_INFO: `/v3/projects/734020a939754d21a86482a3d013cb6d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:57.032962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cb818-47f7-451d-b0e9-43c6fd3cde14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=734020a939754d21a86482a3d013cb6d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:57.036134 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cb818-47f7-451d-b0e9-43c6fd3cde14 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:57.084711 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-571cb818-47f7-451d-b0e9-43c6fd3cde14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '734020a939754d21a86482a3d013cb6d', 'request_id': 'req-571cb818-47f7-451d-b0e9-43c6fd3cde14'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:57.091104 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-571cb818-47f7-451d-b0e9-43c6fd3cde14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '734020a939754d21a86482a3d013cb6d', 'request_id': 'req-571cb818-47f7-451d-b0e9-43c6fd3cde14'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:57.097814 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-571cb818-47f7-451d-b0e9-43c6fd3cde14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '734020a939754d21a86482a3d013cb6d', 'request_id': 'req-571cb818-47f7-451d-b0e9-43c6fd3cde14'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:57.110884 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-571cb818-47f7-451d-b0e9-43c6fd3cde14 admin admin] The token cache is being invalidate because project 734020a939754d21a86482a3d013cb6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:15:57.111173 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-571cb818-47f7-451d-b0e9-43c6fd3cde14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:57.111786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-571cb818-47f7-451d-b0e9-43c6fd3cde14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:15:57.113490 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1399/2767] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:15:57 2022] DELETE /identity/v3/projects/734020a939754d21a86482a3d013cb6d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:15:57.118579 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19b2de-79d0-4249-a968-abe7158a53bd None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:57.119137 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19b2de-79d0-4249-a968-abe7158a53bd None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:57.119247 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19b2de-79d0-4249-a968-abe7158a53bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:57.137327 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-8c19b2de-79d0-4249-a968-abe7158a53bd None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:15:57.138539 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8c19b2de-79d0-4249-a968-abe7158a53bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:57.197282 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1369/2768] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:15:57.205860 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eaf748-e376-49fa-8f45-2552def4072e None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:57.206149 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eaf748-e376-49fa-8f45-2552def4072e None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:57.206429 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eaf748-e376-49fa-8f45-2552def4072e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:57.225404 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-37eaf748-e376-49fa-8f45-2552def4072e None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:15:57.226629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-37eaf748-e376-49fa-8f45-2552def4072e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:57.235955 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1400/2769] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:15:57.240478 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19b2de-79d0-4249-a968-abe7158a53bd None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:57.251126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19b2de-79d0-4249-a968-abe7158a53bd None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:57.255910 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e82d48-5d66-40a3-95b1-a4c950547736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:57.256689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e82d48-5d66-40a3-95b1-a4c950547736 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:57.257081 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e82d48-5d66-40a3-95b1-a4c950547736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:57.257415 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e82d48-5d66-40a3-95b1-a4c950547736 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:57.258245 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e82d48-5d66-40a3-95b1-a4c950547736 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:15:57.258650 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e82d48-5d66-40a3-95b1-a4c950547736 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:57.260432 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e82d48-5d66-40a3-95b1-a4c950547736 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:57.274213 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1370/2770] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:15:57.279297 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eaf748-e376-49fa-8f45-2552def4072e None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:57.289320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eaf748-e376-49fa-8f45-2552def4072e None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:57.295014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e27c79-e2ff-420c-88cc-8b203b25a088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:57.295728 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e27c79-e2ff-420c-88cc-8b203b25a088 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:57.296007 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e27c79-e2ff-420c-88cc-8b203b25a088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:57.296275 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e27c79-e2ff-420c-88cc-8b203b25a088 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:57.297260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e27c79-e2ff-420c-88cc-8b203b25a088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:57.299180 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e27c79-e2ff-420c-88cc-8b203b25a088 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:57.333093 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1401/2771] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:15:57 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:15:57.334220 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e82d48-5d66-40a3-95b1-a4c950547736 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:57.344680 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e82d48-5d66-40a3-95b1-a4c950547736 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:57.370059 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:15:57.412819 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f35ad-4746-4bbc-a3e9-4049770ae945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:57.413501 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f35ad-4746-4bbc-a3e9-4049770ae945 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:57.413784 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f35ad-4746-4bbc-a3e9-4049770ae945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:57.414059 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f35ad-4746-4bbc-a3e9-4049770ae945 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:57.414760 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f35ad-4746-4bbc-a3e9-4049770ae945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:57.416864 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f35ad-4746-4bbc-a3e9-4049770ae945 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:57.442180 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1371/2772] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:15:57 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:15:57.443792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e27c79-e2ff-420c-88cc-8b203b25a088 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:57.455464 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e27c79-e2ff-420c-88cc-8b203b25a088 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:57.460283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532bc1c-b210-40f5-bf97-34b86ea589d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:57.460988 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532bc1c-b210-40f5-bf97-34b86ea589d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:57.461270 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532bc1c-b210-40f5-bf97-34b86ea589d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:57.461588 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532bc1c-b210-40f5-bf97-34b86ea589d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:57.462590 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2532bc1c-b210-40f5-bf97-34b86ea589d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:57.464190 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2532bc1c-b210-40f5-bf97-34b86ea589d5 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:57.465787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:15:57.478886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1402/2773] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:15:57.482693 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f35ad-4746-4bbc-a3e9-4049770ae945 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:57.493434 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f35ad-4746-4bbc-a3e9-4049770ae945 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:57.498337 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b6110-6e91-4a1e-867e-378c370e080a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:57.499004 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b6110-6e91-4a1e-867e-378c370e080a admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:57.499285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b6110-6e91-4a1e-867e-378c370e080a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:57.499561 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b6110-6e91-4a1e-867e-378c370e080a admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:57.500286 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b6110-6e91-4a1e-867e-378c370e080a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:57.502014 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b6110-6e91-4a1e-867e-378c370e080a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:57.513388 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1372/2774] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:15:57.518036 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532bc1c-b210-40f5-bf97-34b86ea589d5 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:57.529634 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532bc1c-b210-40f5-bf97-34b86ea589d5 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:57.534881 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef4744-31bb-4bc3-b029-0deee7017a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:57.535813 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef4744-31bb-4bc3-b029-0deee7017a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:57.536063 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef4744-31bb-4bc3-b029-0deee7017a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:57.536369 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef4744-31bb-4bc3-b029-0deee7017a0c admin admin] PATH_INFO: `/v3/projects/daaea30da7af4734a2a075c041957057/users/ab61a8099b8242e4bd8a2995d3e0239d/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:57.553352 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef4744-31bb-4bc3-b029-0deee7017a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=daaea30da7af4734a2a075c041957057, user_id=ab61a8099b8242e4bd8a2995d3e0239d, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:57.556321 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef4744-31bb-4bc3-b029-0deee7017a0c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:57.570787 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1403/2775] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:15:57 2022] PUT /identity/v3/projects/daaea30da7af4734a2a075c041957057/users/ab61a8099b8242e4bd8a2995d3e0239d/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:15:57.581547 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8afce-e68c-4e18-a710-9ad305e6f23e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:57.581877 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8afce-e68c-4e18-a710-9ad305e6f23e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:57.582179 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8afce-e68c-4e18-a710-9ad305e6f23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:57.613352 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-5bd8afce-e68c-4e18-a710-9ad305e6f23e None None] MFA Rules not processed for user `ab61a8099b8242e4bd8a2995d3e0239d`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:15:57.615257 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5bd8afce-e68c-4e18-a710-9ad305e6f23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:57.723538 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1373/2776] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:15:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:15:57.725003 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef4744-31bb-4bc3-b029-0deee7017a0c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:57.812283 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef4744-31bb-4bc3-b029-0deee7017a0c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:57.819803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199758b-17ac-4043-8437-1f072790991a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:57.820527 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199758b-17ac-4043-8437-1f072790991a admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:57.820819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199758b-17ac-4043-8437-1f072790991a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:57.821098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199758b-17ac-4043-8437-1f072790991a admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:57.822075 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199758b-17ac-4043-8437-1f072790991a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:57.824012 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199758b-17ac-4043-8437-1f072790991a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:57.873441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1404/2777] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:15:57 2022] POST /identity/v3/projects => generated 393 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:15:57.877372 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8afce-e68c-4e18-a710-9ad305e6f23e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:57.894414 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8afce-e68c-4e18-a710-9ad305e6f23e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:57.960763 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570869f4-b381-4b3a-b79b-b1e1728bcdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:57.961485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570869f4-b381-4b3a-b79b-b1e1728bcdca admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:57.962023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570869f4-b381-4b3a-b79b-b1e1728bcdca admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:57.962297 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570869f4-b381-4b3a-b79b-b1e1728bcdca admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:57.963008 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570869f4-b381-4b3a-b79b-b1e1728bcdca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:57.965098 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570869f4-b381-4b3a-b79b-b1e1728bcdca admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:57.991161 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1374/2778] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:15:57 2022] POST /identity/v3/users => generated 303 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:15:57.992575 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199758b-17ac-4043-8437-1f072790991a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:58.005499 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199758b-17ac-4043-8437-1f072790991a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:58.010400 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52fe9e-6612-456a-9ec9-2ebde89d01c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:58.011103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52fe9e-6612-456a-9ec9-2ebde89d01c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:58.011388 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52fe9e-6612-456a-9ec9-2ebde89d01c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:58.011667 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52fe9e-6612-456a-9ec9-2ebde89d01c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:58.012428 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52fe9e-6612-456a-9ec9-2ebde89d01c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:58.014348 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52fe9e-6612-456a-9ec9-2ebde89d01c4 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:58.028627 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1405/2779] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:15:58.029749 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570869f4-b381-4b3a-b79b-b1e1728bcdca admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:58.040363 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570869f4-b381-4b3a-b79b-b1e1728bcdca admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:58.045421 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d4dfc-bcbf-4724-a873-963b84a8c8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:58.046312 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d4dfc-bcbf-4724-a873-963b84a8c8b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:58.046591 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d4dfc-bcbf-4724-a873-963b84a8c8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:58.046863 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d4dfc-bcbf-4724-a873-963b84a8c8b7 admin admin] PATH_INFO: `/v3/projects/4a50f99b20824df5b732c54d38f4a71c/users/a55226dab4d641a69c005b9361065a05/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:58.061797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d4dfc-bcbf-4724-a873-963b84a8c8b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a50f99b20824df5b732c54d38f4a71c, user_id=a55226dab4d641a69c005b9361065a05, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:58.064459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d4dfc-bcbf-4724-a873-963b84a8c8b7 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:58.074075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:15:58.080936 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1375/2780] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:15:58 2022] PUT /identity/v3/projects/4a50f99b20824df5b732c54d38f4a71c/users/a55226dab4d641a69c005b9361065a05/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:15:58.084940 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52fe9e-6612-456a-9ec9-2ebde89d01c4 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:58.096162 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52fe9e-6612-456a-9ec9-2ebde89d01c4 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:58.157044 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2c32a-1c9b-46e5-b38b-6917601bff14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:58.157720 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2c32a-1c9b-46e5-b38b-6917601bff14 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:58.157996 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2c32a-1c9b-46e5-b38b-6917601bff14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:58.158271 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2c32a-1c9b-46e5-b38b-6917601bff14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:58.159004 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2c32a-1c9b-46e5-b38b-6917601bff14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:58.160629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2c32a-1c9b-46e5-b38b-6917601bff14 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:58.171145 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1406/2781] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:15:58.175317 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d4dfc-bcbf-4724-a873-963b84a8c8b7 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:58.186073 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d4dfc-bcbf-4724-a873-963b84a8c8b7 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:58.191354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6f931-eb84-4acf-9b17-8dd61cb02e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:58.192244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6f931-eb84-4acf-9b17-8dd61cb02e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:58.192556 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6f931-eb84-4acf-9b17-8dd61cb02e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:58.192823 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6f931-eb84-4acf-9b17-8dd61cb02e0e admin admin] PATH_INFO: `/v3/projects/4a50f99b20824df5b732c54d38f4a71c/users/a55226dab4d641a69c005b9361065a05/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:58.195058 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6f931-eb84-4acf-9b17-8dd61cb02e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a50f99b20824df5b732c54d38f4a71c, user_id=a55226dab4d641a69c005b9361065a05, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:58.197852 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6f931-eb84-4acf-9b17-8dd61cb02e0e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:58.209409 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1376/2782] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:15:58 2022] PUT /identity/v3/projects/4a50f99b20824df5b732c54d38f4a71c/users/a55226dab4d641a69c005b9361065a05/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:15:58.221003 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3dfd2c-3f3e-488d-8ab5-78bfa56134e0 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:58.221251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3dfd2c-3f3e-488d-8ab5-78bfa56134e0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:58.221559 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3dfd2c-3f3e-488d-8ab5-78bfa56134e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:58.252811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1c3dfd2c-3f3e-488d-8ab5-78bfa56134e0 None None] MFA Rules not processed for user `a55226dab4d641a69c005b9361065a05`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:15:58.254065 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1c3dfd2c-3f3e-488d-8ab5-78bfa56134e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:58.293474 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:15:58.345819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1407/2783] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:15:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:15:58.356946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6f931-eb84-4acf-9b17-8dd61cb02e0e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:58.358651 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-5be6f931-eb84-4acf-9b17-8dd61cb02e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:58.421507 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6f931-eb84-4acf-9b17-8dd61cb02e0e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:58.425334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d9870-776f-452d-8ca8-f4e28ac19b6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:58.425997 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d9870-776f-452d-8ca8-f4e28ac19b6d service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:58.426281 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d9870-776f-452d-8ca8-f4e28ac19b6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:58.426563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d9870-776f-452d-8ca8-f4e28ac19b6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:58.438631 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d9870-776f-452d-8ca8-f4e28ac19b6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:58.440527 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d9870-776f-452d-8ca8-f4e28ac19b6d service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:58.451931 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1377/2784] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:15:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:15:58.499848 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3dfd2c-3f3e-488d-8ab5-78bfa56134e0 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:15:58.501440 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1c3dfd2c-3f3e-488d-8ab5-78bfa56134e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:58.514793 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3dfd2c-3f3e-488d-8ab5-78bfa56134e0 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:15:58.518995 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85f688-9502-4d21-829c-9ac75b5b6b16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:15:58.519652 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85f688-9502-4d21-829c-9ac75b5b6b16 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:58.519930 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85f688-9502-4d21-829c-9ac75b5b6b16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:58.520202 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85f688-9502-4d21-829c-9ac75b5b6b16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:58.584099 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85f688-9502-4d21-829c-9ac75b5b6b16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:15:58.585623 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85f688-9502-4d21-829c-9ac75b5b6b16 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:15:58.596516 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1408/2785] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:15:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:15:58.898896 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f49d9f-9bc8-4c99-b2fc-2a5fb8aa5f24 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:15:58.899185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f49d9f-9bc8-4c99-b2fc-2a5fb8aa5f24 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:15:58.899516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f49d9f-9bc8-4c99-b2fc-2a5fb8aa5f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:15:58.920151 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-18f49d9f-9bc8-4c99-b2fc-2a5fb8aa5f24 None None] MFA Rules not processed for user `0ef599055aed42f3a97fb30bba553fb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:15:58.921430 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-18f49d9f-9bc8-4c99-b2fc-2a5fb8aa5f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:15:58.981525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1378/2786] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:15:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:01.994717 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85f688-9502-4d21-829c-9ac75b5b6b16 service cinder] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:02.069604 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85f688-9502-4d21-829c-9ac75b5b6b16 service cinder] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:02.074643 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f3cf6-1354-4edd-bdb0-52a7ada76818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:02.075586 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f3cf6-1354-4edd-bdb0-52a7ada76818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:02.075869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f3cf6-1354-4edd-bdb0-52a7ada76818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:02.076214 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f3cf6-1354-4edd-bdb0-52a7ada76818 admin admin] PATH_INFO: `/v3/users/ab61a8099b8242e4bd8a2995d3e0239d` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:02.077654 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f3cf6-1354-4edd-bdb0-52a7ada76818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab61a8099b8242e4bd8a2995d3e0239d)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:02.079595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f3cf6-1354-4edd-bdb0-52a7ada76818 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:02.129081 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-762f3cf6-1354-4edd-bdb0-52a7ada76818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab61a8099b8242e4bd8a2995d3e0239d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:02.136813 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-762f3cf6-1354-4edd-bdb0-52a7ada76818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab61a8099b8242e4bd8a2995d3e0239d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:02.137477 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-762f3cf6-1354-4edd-bdb0-52a7ada76818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab61a8099b8242e4bd8a2995d3e0239d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:02.146856 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-762f3cf6-1354-4edd-bdb0-52a7ada76818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab61a8099b8242e4bd8a2995d3e0239d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:02.147709 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-762f3cf6-1354-4edd-bdb0-52a7ada76818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab61a8099b8242e4bd8a2995d3e0239d'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:02.153468 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:02.170087 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1409/2787] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:16:01 2022] DELETE /identity/v3/users/ab61a8099b8242e4bd8a2995d3e0239d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:02.175549 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41b92c-0390-4b96-a37e-82195730d79a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:02.176710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41b92c-0390-4b96-a37e-82195730d79a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:02.177306 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41b92c-0390-4b96-a37e-82195730d79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:02.196758 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-7c41b92c-0390-4b96-a37e-82195730d79a None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:02.197970 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7c41b92c-0390-4b96-a37e-82195730d79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:02.260385 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1379/2788] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:02.270723 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f3cf6-1354-4edd-bdb0-52a7ada76818 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:02.272139 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-762f3cf6-1354-4edd-bdb0-52a7ada76818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:02.353296 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f3cf6-1354-4edd-bdb0-52a7ada76818 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:02.356919 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12227ffb-6cc2-43db-be92-8795cd8604d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:02.357572 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12227ffb-6cc2-43db-be92-8795cd8604d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:02.358039 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12227ffb-6cc2-43db-be92-8795cd8604d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:02.358406 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12227ffb-6cc2-43db-be92-8795cd8604d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:02.370421 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12227ffb-6cc2-43db-be92-8795cd8604d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:02.371837 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12227ffb-6cc2-43db-be92-8795cd8604d1 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:02.385698 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1410/2789] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:16:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:16:02.748334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41b92c-0390-4b96-a37e-82195730d79a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:02.750040 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7c41b92c-0390-4b96-a37e-82195730d79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:02.763428 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41b92c-0390-4b96-a37e-82195730d79a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:02.784553 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f4611d-af83-4575-a0f6-e9f09f18ba71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:02.785710 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4611d-af83-4575-a0f6-e9f09f18ba71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:02.786082 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4611d-af83-4575-a0f6-e9f09f18ba71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:02.786388 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4611d-af83-4575-a0f6-e9f09f18ba71 admin admin] PATH_INFO: `/v3/projects/daaea30da7af4734a2a075c041957057` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:02.787711 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f4611d-af83-4575-a0f6-e9f09f18ba71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daaea30da7af4734a2a075c041957057)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:02.793643 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f4611d-af83-4575-a0f6-e9f09f18ba71 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:02.842498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:02.862223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b5f4611d-af83-4575-a0f6-e9f09f18ba71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daaea30da7af4734a2a075c041957057', 'request_id': 'req-b5f4611d-af83-4575-a0f6-e9f09f18ba71'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:02.868184 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b5f4611d-af83-4575-a0f6-e9f09f18ba71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daaea30da7af4734a2a075c041957057', 'request_id': 'req-b5f4611d-af83-4575-a0f6-e9f09f18ba71'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:02.874736 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b5f4611d-af83-4575-a0f6-e9f09f18ba71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daaea30da7af4734a2a075c041957057', 'request_id': 'req-b5f4611d-af83-4575-a0f6-e9f09f18ba71'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:02.888971 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b5f4611d-af83-4575-a0f6-e9f09f18ba71 admin admin] The token cache is being invalidate because project daaea30da7af4734a2a075c041957057 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:16:02.889263 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b5f4611d-af83-4575-a0f6-e9f09f18ba71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:02.889912 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-b5f4611d-af83-4575-a0f6-e9f09f18ba71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:02.891495 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1380/2790] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:16:02 2022] DELETE /identity/v3/projects/daaea30da7af4734a2a075c041957057 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:02.895538 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12227ffb-6cc2-43db-be92-8795cd8604d1 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:02.907392 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12227ffb-6cc2-43db-be92-8795cd8604d1 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:02.972577 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06fa43-26d2-445b-8b86-5b3947f2093a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:02.973483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06fa43-26d2-445b-8b86-5b3947f2093a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:02.973767 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06fa43-26d2-445b-8b86-5b3947f2093a admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:02.974040 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06fa43-26d2-445b-8b86-5b3947f2093a admin admin] PATH_INFO: `/v3/users/a55226dab4d641a69c005b9361065a05` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:02.975231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a06fa43-26d2-445b-8b86-5b3947f2093a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a55226dab4d641a69c005b9361065a05)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:02.977175 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a06fa43-26d2-445b-8b86-5b3947f2093a admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:03.029891 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2a06fa43-26d2-445b-8b86-5b3947f2093a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a55226dab4d641a69c005b9361065a05'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:03.037039 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2a06fa43-26d2-445b-8b86-5b3947f2093a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a55226dab4d641a69c005b9361065a05'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:03.037719 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2a06fa43-26d2-445b-8b86-5b3947f2093a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a55226dab4d641a69c005b9361065a05'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:03.046561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2a06fa43-26d2-445b-8b86-5b3947f2093a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a55226dab4d641a69c005b9361065a05'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:03.047226 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-2a06fa43-26d2-445b-8b86-5b3947f2093a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a55226dab4d641a69c005b9361065a05'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:03.066183 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1411/2791] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:16:02 2022] DELETE /identity/v3/users/a55226dab4d641a69c005b9361065a05 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:03.377205 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f4611d-af83-4575-a0f6-e9f09f18ba71 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:03.378712 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b5f4611d-af83-4575-a0f6-e9f09f18ba71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:03.393367 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f4611d-af83-4575-a0f6-e9f09f18ba71 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:03.400483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06fa43-26d2-445b-8b86-5b3947f2093a admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:03.403003 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2a06fa43-26d2-445b-8b86-5b3947f2093a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:03.415107 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:03.466704 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d439b-5fa8-414e-b403-192ae991757a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:03.467885 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d439b-5fa8-414e-b403-192ae991757a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:03.468337 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d439b-5fa8-414e-b403-192ae991757a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:03.468805 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d439b-5fa8-414e-b403-192ae991757a admin admin] PATH_INFO: `/v3/projects/4a50f99b20824df5b732c54d38f4a71c` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:03.470344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d439b-5fa8-414e-b403-192ae991757a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a50f99b20824df5b732c54d38f4a71c)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:03.472804 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d439b-5fa8-414e-b403-192ae991757a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:03.475473 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06fa43-26d2-445b-8b86-5b3947f2093a admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:03.481591 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8fc2de-0cbc-4e80-83b6-5cb1e39b5ea0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:03.482279 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fc2de-0cbc-4e80-83b6-5cb1e39b5ea0 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:03.482651 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fc2de-0cbc-4e80-83b6-5cb1e39b5ea0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:03.482861 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fc2de-0cbc-4e80-83b6-5cb1e39b5ea0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:03.485466 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-0f8fc2de-0cbc-4e80-83b6-5cb1e39b5ea0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:03.529192 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-949d439b-5fa8-414e-b403-192ae991757a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a50f99b20824df5b732c54d38f4a71c', 'request_id': 'req-949d439b-5fa8-414e-b403-192ae991757a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:03.535660 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-949d439b-5fa8-414e-b403-192ae991757a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a50f99b20824df5b732c54d38f4a71c', 'request_id': 'req-949d439b-5fa8-414e-b403-192ae991757a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:03.544681 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-949d439b-5fa8-414e-b403-192ae991757a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a50f99b20824df5b732c54d38f4a71c', 'request_id': 'req-949d439b-5fa8-414e-b403-192ae991757a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:03.559616 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-949d439b-5fa8-414e-b403-192ae991757a admin admin] The token cache is being invalidate because project 4a50f99b20824df5b732c54d38f4a71c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:16:03.559903 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-949d439b-5fa8-414e-b403-192ae991757a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:03.560810 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-949d439b-5fa8-414e-b403-192ae991757a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:03.562628 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1381/2792] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:16:03 2022] DELETE /identity/v3/projects/4a50f99b20824df5b732c54d38f4a71c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:03.567702 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3ab47-1787-4de5-b0dd-c1c4fb1fd83a None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:03.568034 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3ab47-1787-4de5-b0dd-c1c4fb1fd83a None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:03.568343 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3ab47-1787-4de5-b0dd-c1c4fb1fd83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:03.577413 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8fc2de-0cbc-4e80-83b6-5cb1e39b5ea0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:03.578907 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8fc2de-0cbc-4e80-83b6-5cb1e39b5ea0 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:03.590214 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-acb3ab47-1787-4de5-b0dd-c1c4fb1fd83a None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:03.591673 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-acb3ab47-1787-4de5-b0dd-c1c4fb1fd83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:03.592735 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1412/2793] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:16:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:16:03.641358 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:03.663406 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1382/2794] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:03.671892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94ee57-0dd0-4f83-b6b6-f92b0e3c4391 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:03.672255 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94ee57-0dd0-4f83-b6b6-f92b0e3c4391 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:03.672671 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94ee57-0dd0-4f83-b6b6-f92b0e3c4391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:03.691479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-4c94ee57-0dd0-4f83-b6b6-f92b0e3c4391 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:03.692836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4c94ee57-0dd0-4f83-b6b6-f92b0e3c4391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:03.702035 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1413/2795] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:03.706473 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3ab47-1787-4de5-b0dd-c1c4fb1fd83a None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:03.718444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3ab47-1787-4de5-b0dd-c1c4fb1fd83a None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:03.723241 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8690fa-07f6-41fb-8864-2f81c3424185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:03.723886 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8690fa-07f6-41fb-8864-2f81c3424185 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:03.724176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8690fa-07f6-41fb-8864-2f81c3424185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:03.724458 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8690fa-07f6-41fb-8864-2f81c3424185 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:03.725197 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8690fa-07f6-41fb-8864-2f81c3424185 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:16:03.725498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8690fa-07f6-41fb-8864-2f81c3424185 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:03.727095 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8690fa-07f6-41fb-8864-2f81c3424185 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:03.744157 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1383/2796] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:16:03.745742 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94ee57-0dd0-4f83-b6b6-f92b0e3c4391 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:03.755639 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94ee57-0dd0-4f83-b6b6-f92b0e3c4391 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:03.760983 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce81927-e130-4f9f-8ea3-8e24ab4de815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:03.761691 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce81927-e130-4f9f-8ea3-8e24ab4de815 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:03.761971 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce81927-e130-4f9f-8ea3-8e24ab4de815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:03.762245 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce81927-e130-4f9f-8ea3-8e24ab4de815 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:03.762937 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce81927-e130-4f9f-8ea3-8e24ab4de815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:03.765205 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce81927-e130-4f9f-8ea3-8e24ab4de815 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:03.796581 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1414/2797] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:16:03 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:16:03.800857 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8690fa-07f6-41fb-8864-2f81c3424185 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:03.811324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8690fa-07f6-41fb-8864-2f81c3424185 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:03.888127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6598009-1217-4b58-8d3d-127d12389a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:03.889183 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6598009-1217-4b58-8d3d-127d12389a10 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:03.889462 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6598009-1217-4b58-8d3d-127d12389a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:03.889755 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6598009-1217-4b58-8d3d-127d12389a10 admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:03.890508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6598009-1217-4b58-8d3d-127d12389a10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:03.892544 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6598009-1217-4b58-8d3d-127d12389a10 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:03.913818 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1384/2798] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:16:03 2022] POST /identity/v3/users => generated 297 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:16:03.918669 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce81927-e130-4f9f-8ea3-8e24ab4de815 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:03.931772 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce81927-e130-4f9f-8ea3-8e24ab4de815 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:03.937549 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5b68d-7c74-4a33-9f94-6f5f3463afe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:03.938288 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5b68d-7c74-4a33-9f94-6f5f3463afe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:03.938566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5b68d-7c74-4a33-9f94-6f5f3463afe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:03.938835 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5b68d-7c74-4a33-9f94-6f5f3463afe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:03.939569 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce5b68d-7c74-4a33-9f94-6f5f3463afe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:03.941254 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce5b68d-7c74-4a33-9f94-6f5f3463afe9 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:03.952259 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1415/2799] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:16:03.956813 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6598009-1217-4b58-8d3d-127d12389a10 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:03.970943 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6598009-1217-4b58-8d3d-127d12389a10 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:03.977354 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb2024-664b-4a80-8d90-0fd4488ba3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:03.977989 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb2024-664b-4a80-8d90-0fd4488ba3c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:03.978284 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb2024-664b-4a80-8d90-0fd4488ba3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:03.978559 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb2024-664b-4a80-8d90-0fd4488ba3c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:03.979287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb2024-664b-4a80-8d90-0fd4488ba3c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:03.980946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb2024-664b-4a80-8d90-0fd4488ba3c3 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:03.991762 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1385/2800] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:16:03.996548 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5b68d-7c74-4a33-9f94-6f5f3463afe9 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:04.010040 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5b68d-7c74-4a33-9f94-6f5f3463afe9 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:04.016664 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51ed94-f7fb-430c-90d3-efe9253b7e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:04.018214 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51ed94-f7fb-430c-90d3-efe9253b7e92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:04.018621 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51ed94-f7fb-430c-90d3-efe9253b7e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:04.019009 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51ed94-f7fb-430c-90d3-efe9253b7e92 admin admin] PATH_INFO: `/v3/projects/e527af9790614f38878dec8b4f8d4687/users/b83b88b959b54c109a3beb0bf1923c15/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:04.041119 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51ed94-f7fb-430c-90d3-efe9253b7e92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e527af9790614f38878dec8b4f8d4687, user_id=b83b88b959b54c109a3beb0bf1923c15, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:04.043811 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51ed94-f7fb-430c-90d3-efe9253b7e92 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:04.059422 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1416/2801] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:16:03 2022] PUT /identity/v3/projects/e527af9790614f38878dec8b4f8d4687/users/b83b88b959b54c109a3beb0bf1923c15/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:04.069637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6c010-0c4d-44cc-af96-1d7657fa4bf1 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:04.069933 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6c010-0c4d-44cc-af96-1d7657fa4bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:04.070235 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6c010-0c4d-44cc-af96-1d7657fa4bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:04.100503 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-14f6c010-0c4d-44cc-af96-1d7657fa4bf1 None None] MFA Rules not processed for user `b83b88b959b54c109a3beb0bf1923c15`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:04.102245 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-14f6c010-0c4d-44cc-af96-1d7657fa4bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:04.172609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1386/2802] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:04.177239 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51ed94-f7fb-430c-90d3-efe9253b7e92 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:04.230436 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:04.250836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51ed94-f7fb-430c-90d3-efe9253b7e92 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:04.256086 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36855a91-ea2d-42b1-a2d7-8ab89b48d00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:04.256778 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36855a91-ea2d-42b1-a2d7-8ab89b48d00c admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:04.257058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36855a91-ea2d-42b1-a2d7-8ab89b48d00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:04.257312 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36855a91-ea2d-42b1-a2d7-8ab89b48d00c admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:04.258001 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36855a91-ea2d-42b1-a2d7-8ab89b48d00c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:04.260081 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36855a91-ea2d-42b1-a2d7-8ab89b48d00c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:04.289393 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1417/2803] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:16:04 2022] POST /identity/v3/projects => generated 401 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:16:04.294498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6c010-0c4d-44cc-af96-1d7657fa4bf1 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:04.306508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6c010-0c4d-44cc-af96-1d7657fa4bf1 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:04.347914 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:04.365587 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012324ba-b89c-42a9-a21f-c90bbea8c6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:04.366264 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012324ba-b89c-42a9-a21f-c90bbea8c6fc admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:04.366565 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012324ba-b89c-42a9-a21f-c90bbea8c6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:04.366822 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012324ba-b89c-42a9-a21f-c90bbea8c6fc admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:04.367524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012324ba-b89c-42a9-a21f-c90bbea8c6fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:04.369530 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012324ba-b89c-42a9-a21f-c90bbea8c6fc admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:04.391835 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1387/2804] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:16:04 2022] POST /identity/v3/users => generated 307 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:16:04.395986 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36855a91-ea2d-42b1-a2d7-8ab89b48d00c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:04.407609 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36855a91-ea2d-42b1-a2d7-8ab89b48d00c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:04.413057 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8eeec4-16d7-4ceb-a8c2-8d53030d9968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:04.413726 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8eeec4-16d7-4ceb-a8c2-8d53030d9968 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:04.414011 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8eeec4-16d7-4ceb-a8c2-8d53030d9968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:04.414291 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8eeec4-16d7-4ceb-a8c2-8d53030d9968 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:04.415097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8eeec4-16d7-4ceb-a8c2-8d53030d9968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:04.416753 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8eeec4-16d7-4ceb-a8c2-8d53030d9968 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:04.427624 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1418/2805] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:16:04.432773 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012324ba-b89c-42a9-a21f-c90bbea8c6fc admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:04.444083 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012324ba-b89c-42a9-a21f-c90bbea8c6fc admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:04.449854 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacd114-bd58-4310-9b71-9c524c9053cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:04.450749 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacd114-bd58-4310-9b71-9c524c9053cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:04.451026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacd114-bd58-4310-9b71-9c524c9053cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:04.451301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacd114-bd58-4310-9b71-9c524c9053cc admin admin] PATH_INFO: `/v3/projects/b3c922439ccc4611a0b7ddbadbbfe56e/users/a055e9bf51b4449a8badac2701443888/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:04.468544 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aacd114-bd58-4310-9b71-9c524c9053cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c922439ccc4611a0b7ddbadbbfe56e, user_id=a055e9bf51b4449a8badac2701443888, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:04.471305 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aacd114-bd58-4310-9b71-9c524c9053cc admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:04.491459 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1388/2806] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:16:04 2022] PUT /identity/v3/projects/b3c922439ccc4611a0b7ddbadbbfe56e/users/a055e9bf51b4449a8badac2701443888/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:04.497520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8eeec4-16d7-4ceb-a8c2-8d53030d9968 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:04.512118 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8eeec4-16d7-4ceb-a8c2-8d53030d9968 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:04.573972 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e635eaf-f6df-4d52-a5cf-56eaf7f20d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:04.574626 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e635eaf-f6df-4d52-a5cf-56eaf7f20d99 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:04.574909 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e635eaf-f6df-4d52-a5cf-56eaf7f20d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:04.575163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e635eaf-f6df-4d52-a5cf-56eaf7f20d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:04.575884 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e635eaf-f6df-4d52-a5cf-56eaf7f20d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:04.577508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e635eaf-f6df-4d52-a5cf-56eaf7f20d99 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:04.587632 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1419/2807] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:16:04.592040 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacd114-bd58-4310-9b71-9c524c9053cc admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:04.603552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacd114-bd58-4310-9b71-9c524c9053cc admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:04.608861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b7043-1fd1-432a-9efd-dd46d6fd895b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:04.609738 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b7043-1fd1-432a-9efd-dd46d6fd895b admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:04.610011 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b7043-1fd1-432a-9efd-dd46d6fd895b admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:04.610278 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b7043-1fd1-432a-9efd-dd46d6fd895b admin admin] PATH_INFO: `/v3/projects/b3c922439ccc4611a0b7ddbadbbfe56e/users/a055e9bf51b4449a8badac2701443888/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:04.612359 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b7043-1fd1-432a-9efd-dd46d6fd895b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3c922439ccc4611a0b7ddbadbbfe56e, user_id=a055e9bf51b4449a8badac2701443888, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:04.615090 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b7043-1fd1-432a-9efd-dd46d6fd895b admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:04.626156 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1389/2808] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:16:04 2022] PUT /identity/v3/projects/b3c922439ccc4611a0b7ddbadbbfe56e/users/a055e9bf51b4449a8badac2701443888/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:04.640820 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4e336-7798-4b04-8ad8-d769bd5af937 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:04.641103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4e336-7798-4b04-8ad8-d769bd5af937 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:04.641381 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4e336-7798-4b04-8ad8-d769bd5af937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:04.669456 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-b7a4e336-7798-4b04-8ad8-d769bd5af937 None None] MFA Rules not processed for user `a055e9bf51b4449a8badac2701443888`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:04.670661 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b7a4e336-7798-4b04-8ad8-d769bd5af937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:04.760497 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1420/2809] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:04 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:04.761186 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b7043-1fd1-432a-9efd-dd46d6fd895b admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:04.763181 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ec2b7043-1fd1-432a-9efd-dd46d6fd895b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:04.851457 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b7043-1fd1-432a-9efd-dd46d6fd895b admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:04.856226 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bd30c-760e-4241-896e-421df5efc4d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:04.856915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bd30c-760e-4241-896e-421df5efc4d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:04.857193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bd30c-760e-4241-896e-421df5efc4d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:04.857465 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bd30c-760e-4241-896e-421df5efc4d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:04.867444 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bd30c-760e-4241-896e-421df5efc4d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:04.869031 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bd30c-760e-4241-896e-421df5efc4d3 service cinder] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:04.879276 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1390/2810] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:16:04 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:16:04.954191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4e336-7798-4b04-8ad8-d769bd5af937 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:04.964561 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4e336-7798-4b04-8ad8-d769bd5af937 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:04.968258 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea796d-b84b-4747-b4cd-e4fe12b3497c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:04.968939 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea796d-b84b-4747-b4cd-e4fe12b3497c service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:04.969224 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea796d-b84b-4747-b4cd-e4fe12b3497c service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:04.969501 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea796d-b84b-4747-b4cd-e4fe12b3497c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:05.012387 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:05.032761 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea796d-b84b-4747-b4cd-e4fe12b3497c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:05.034212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea796d-b84b-4747-b4cd-e4fe12b3497c service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:05.044009 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1421/2811] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:16:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:16:09.165334 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bd30c-760e-4241-896e-421df5efc4d3 service cinder] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:09.231201 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bd30c-760e-4241-896e-421df5efc4d3 service cinder] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:09.235953 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23997d7b-31ad-4b17-a750-613e3e9acca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:09.236972 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23997d7b-31ad-4b17-a750-613e3e9acca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:09.237254 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23997d7b-31ad-4b17-a750-613e3e9acca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:09.237523 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23997d7b-31ad-4b17-a750-613e3e9acca4 admin admin] PATH_INFO: `/v3/users/b83b88b959b54c109a3beb0bf1923c15` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:09.238737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23997d7b-31ad-4b17-a750-613e3e9acca4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b83b88b959b54c109a3beb0bf1923c15)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:09.240682 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23997d7b-31ad-4b17-a750-613e3e9acca4 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:09.286075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:09.293277 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-23997d7b-31ad-4b17-a750-613e3e9acca4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b83b88b959b54c109a3beb0bf1923c15'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:09.294396 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-23997d7b-31ad-4b17-a750-613e3e9acca4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b83b88b959b54c109a3beb0bf1923c15'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:09.304384 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-23997d7b-31ad-4b17-a750-613e3e9acca4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b83b88b959b54c109a3beb0bf1923c15'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:09.311528 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-23997d7b-31ad-4b17-a750-613e3e9acca4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b83b88b959b54c109a3beb0bf1923c15'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:09.312168 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-23997d7b-31ad-4b17-a750-613e3e9acca4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b83b88b959b54c109a3beb0bf1923c15'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:09.330656 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1391/2812] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:16:09 2022] DELETE /identity/v3/users/b83b88b959b54c109a3beb0bf1923c15 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:09.335565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0c522-2829-4540-b957-6b2d2c485d00 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:09.336090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0c522-2829-4540-b957-6b2d2c485d00 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:09.336090 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0c522-2829-4540-b957-6b2d2c485d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:09.354975 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-8ac0c522-2829-4540-b957-6b2d2c485d00 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:09.356311 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8ac0c522-2829-4540-b957-6b2d2c485d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:09.431260 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1422/2813] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:09.440333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23997d7b-31ad-4b17-a750-613e3e9acca4 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:09.442079 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-23997d7b-31ad-4b17-a750-613e3e9acca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:09.503500 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23997d7b-31ad-4b17-a750-613e3e9acca4 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:09.507998 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5545f39-aca9-4fef-b4e7-6d70cc175f55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:09.508806 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5545f39-aca9-4fef-b4e7-6d70cc175f55 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:09.509268 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5545f39-aca9-4fef-b4e7-6d70cc175f55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:09.509540 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5545f39-aca9-4fef-b4e7-6d70cc175f55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:09.520061 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5545f39-aca9-4fef-b4e7-6d70cc175f55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:09.521525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5545f39-aca9-4fef-b4e7-6d70cc175f55 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:09.531600 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1392/2814] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:16:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:16:09.629535 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0c522-2829-4540-b957-6b2d2c485d00 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:09.631103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-8ac0c522-2829-4540-b957-6b2d2c485d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:09.645261 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0c522-2829-4540-b957-6b2d2c485d00 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:09.649080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d56523-28d1-4c65-96be-6bd585438626 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:09.649809 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d56523-28d1-4c65-96be-6bd585438626 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:09.650096 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d56523-28d1-4c65-96be-6bd585438626 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:09.650380 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d56523-28d1-4c65-96be-6bd585438626 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:09.652243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a5d56523-28d1-4c65-96be-6bd585438626 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:09.732978 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d56523-28d1-4c65-96be-6bd585438626 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:09.734605 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d56523-28d1-4c65-96be-6bd585438626 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:09.746869 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1423/2815] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:16:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:16:09.813521 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5545f39-aca9-4fef-b4e7-6d70cc175f55 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:09.815020 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e5545f39-aca9-4fef-b4e7-6d70cc175f55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:09.835378 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5545f39-aca9-4fef-b4e7-6d70cc175f55 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:09.842365 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94128351-9773-45c7-9605-55975088a9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:09.843233 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94128351-9773-45c7-9605-55975088a9fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:09.843514 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94128351-9773-45c7-9605-55975088a9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:09.843789 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94128351-9773-45c7-9605-55975088a9fe admin admin] PATH_INFO: `/v3/projects/e527af9790614f38878dec8b4f8d4687` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:09.852724 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94128351-9773-45c7-9605-55975088a9fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e527af9790614f38878dec8b4f8d4687)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:09.855103 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94128351-9773-45c7-9605-55975088a9fe admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:09.906359 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-94128351-9773-45c7-9605-55975088a9fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e527af9790614f38878dec8b4f8d4687', 'request_id': 'req-94128351-9773-45c7-9605-55975088a9fe'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:09.911639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-94128351-9773-45c7-9605-55975088a9fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e527af9790614f38878dec8b4f8d4687', 'request_id': 'req-94128351-9773-45c7-9605-55975088a9fe'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:09.918034 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-94128351-9773-45c7-9605-55975088a9fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e527af9790614f38878dec8b4f8d4687', 'request_id': 'req-94128351-9773-45c7-9605-55975088a9fe'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:09.935551 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-94128351-9773-45c7-9605-55975088a9fe admin admin] The token cache is being invalidate because project e527af9790614f38878dec8b4f8d4687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:16:09.935795 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-94128351-9773-45c7-9605-55975088a9fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:09.936498 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-94128351-9773-45c7-9605-55975088a9fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:09.938161 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1393/2816] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:16:09 2022] DELETE /identity/v3/projects/e527af9790614f38878dec8b4f8d4687 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:09.942242 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d56523-28d1-4c65-96be-6bd585438626 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:09.953398 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d56523-28d1-4c65-96be-6bd585438626 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:10.017938 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510862e-62b2-4fc5-8c1e-d2f8c273fc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:10.018825 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510862e-62b2-4fc5-8c1e-d2f8c273fc90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:10.019101 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510862e-62b2-4fc5-8c1e-d2f8c273fc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:10.019371 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510862e-62b2-4fc5-8c1e-d2f8c273fc90 admin admin] PATH_INFO: `/v3/users/a055e9bf51b4449a8badac2701443888` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:10.020733 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510862e-62b2-4fc5-8c1e-d2f8c273fc90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a055e9bf51b4449a8badac2701443888)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:10.022652 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510862e-62b2-4fc5-8c1e-d2f8c273fc90 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:10.027599 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=100 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:10.077436 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c510862e-62b2-4fc5-8c1e-d2f8c273fc90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a055e9bf51b4449a8badac2701443888'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:10.084855 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c510862e-62b2-4fc5-8c1e-d2f8c273fc90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a055e9bf51b4449a8badac2701443888'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:10.085611 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c510862e-62b2-4fc5-8c1e-d2f8c273fc90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a055e9bf51b4449a8badac2701443888'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:10.093991 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c510862e-62b2-4fc5-8c1e-d2f8c273fc90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a055e9bf51b4449a8badac2701443888'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:10.094587 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-c510862e-62b2-4fc5-8c1e-d2f8c273fc90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a055e9bf51b4449a8badac2701443888'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:10.112431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1424/2817] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:16:09 2022] DELETE /identity/v3/users/a055e9bf51b4449a8badac2701443888 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:10.423169 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94128351-9773-45c7-9605-55975088a9fe admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:10.424700 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-94128351-9773-45c7-9605-55975088a9fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:10.439621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94128351-9773-45c7-9605-55975088a9fe admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:10.481691 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:10.503402 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd26b4-e594-42e0-b267-ca57ccd5578a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:10.504608 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd26b4-e594-42e0-b267-ca57ccd5578a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:10.504945 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd26b4-e594-42e0-b267-ca57ccd5578a admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:10.505244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd26b4-e594-42e0-b267-ca57ccd5578a admin admin] PATH_INFO: `/v3/projects/b3c922439ccc4611a0b7ddbadbbfe56e` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:10.507109 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfd26b4-e594-42e0-b267-ca57ccd5578a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3c922439ccc4611a0b7ddbadbbfe56e)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:10.509613 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfd26b4-e594-42e0-b267-ca57ccd5578a admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:10.563381 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9cfd26b4-e594-42e0-b267-ca57ccd5578a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3c922439ccc4611a0b7ddbadbbfe56e', 'request_id': 'req-9cfd26b4-e594-42e0-b267-ca57ccd5578a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:10.569435 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9cfd26b4-e594-42e0-b267-ca57ccd5578a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3c922439ccc4611a0b7ddbadbbfe56e', 'request_id': 'req-9cfd26b4-e594-42e0-b267-ca57ccd5578a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:10.575945 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9cfd26b4-e594-42e0-b267-ca57ccd5578a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3c922439ccc4611a0b7ddbadbbfe56e', 'request_id': 'req-9cfd26b4-e594-42e0-b267-ca57ccd5578a'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:10.591545 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9cfd26b4-e594-42e0-b267-ca57ccd5578a admin admin] The token cache is being invalidate because project b3c922439ccc4611a0b7ddbadbbfe56e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:16:10.591995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9cfd26b4-e594-42e0-b267-ca57ccd5578a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:10.592685 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-9cfd26b4-e594-42e0-b267-ca57ccd5578a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:10.594285 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1394/2818] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:16:10 2022] DELETE /identity/v3/projects/b3c922439ccc4611a0b7ddbadbbfe56e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:10.599394 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9f2a7-5500-4719-aebd-a564156ca58a None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:10.599886 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9f2a7-5500-4719-aebd-a564156ca58a None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:10.600712 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9f2a7-5500-4719-aebd-a564156ca58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:10.620378 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-47b9f2a7-5500-4719-aebd-a564156ca58a None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:10.621636 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-47b9f2a7-5500-4719-aebd-a564156ca58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:10.686675 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1425/2819] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:10.694523 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29625662-a33c-42cf-8b8c-c4f709caa28e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:10.694808 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29625662-a33c-42cf-8b8c-c4f709caa28e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:10.695082 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29625662-a33c-42cf-8b8c-c4f709caa28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:10.715865 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-29625662-a33c-42cf-8b8c-c4f709caa28e None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:10.717164 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-29625662-a33c-42cf-8b8c-c4f709caa28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:10.729222 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1395/2820] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:10.733937 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9f2a7-5500-4719-aebd-a564156ca58a None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:10.743791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9f2a7-5500-4719-aebd-a564156ca58a None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:10.748583 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f058f11-09e7-4bc8-9b0c-500d8eaf0a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:10.749273 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f058f11-09e7-4bc8-9b0c-500d8eaf0a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:10.749540 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f058f11-09e7-4bc8-9b0c-500d8eaf0a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:10.749815 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f058f11-09e7-4bc8-9b0c-500d8eaf0a3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:10.750548 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f058f11-09e7-4bc8-9b0c-500d8eaf0a3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80124) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:16:10.750857 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f058f11-09e7-4bc8-9b0c-500d8eaf0a3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:10.752568 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f058f11-09e7-4bc8-9b0c-500d8eaf0a3f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:10.765777 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1426/2821] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:16:10.770057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29625662-a33c-42cf-8b8c-c4f709caa28e None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:10.780023 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29625662-a33c-42cf-8b8c-c4f709caa28e None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:10.785412 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a609b59-00cd-4b11-9303-b2dfc6714af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:10.786102 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a609b59-00cd-4b11-9303-b2dfc6714af5 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:10.786381 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a609b59-00cd-4b11-9303-b2dfc6714af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:10.786651 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a609b59-00cd-4b11-9303-b2dfc6714af5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:10.787418 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a609b59-00cd-4b11-9303-b2dfc6714af5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:10.789484 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a609b59-00cd-4b11-9303-b2dfc6714af5 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:10.820287 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1396/2822] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:16:10 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:16:10.824841 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f058f11-09e7-4bc8-9b0c-500d8eaf0a3f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:10.835479 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f058f11-09e7-4bc8-9b0c-500d8eaf0a3f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:10.903165 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831e4dd-ccfc-45a4-bdaa-393a04155f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:10.903865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831e4dd-ccfc-45a4-bdaa-393a04155f34 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:10.904163 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831e4dd-ccfc-45a4-bdaa-393a04155f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:10.904439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831e4dd-ccfc-45a4-bdaa-393a04155f34 admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:10.905155 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831e4dd-ccfc-45a4-bdaa-393a04155f34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:10.907034 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831e4dd-ccfc-45a4-bdaa-393a04155f34 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:10.932710 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:10.940199 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1427/2823] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:16:10 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:16:10.943350 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a609b59-00cd-4b11-9303-b2dfc6714af5 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:10.957945 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a609b59-00cd-4b11-9303-b2dfc6714af5 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:10.964164 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be134c09-01b9-400d-ae6c-e3bdddcde56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:10.964886 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be134c09-01b9-400d-ae6c-e3bdddcde56f admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:10.965168 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be134c09-01b9-400d-ae6c-e3bdddcde56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:10.965998 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be134c09-01b9-400d-ae6c-e3bdddcde56f admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:10.966381 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be134c09-01b9-400d-ae6c-e3bdddcde56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:10.968236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be134c09-01b9-400d-ae6c-e3bdddcde56f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:10.980644 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1397/2824] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:16:10.984538 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831e4dd-ccfc-45a4-bdaa-393a04155f34 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:10.995212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831e4dd-ccfc-45a4-bdaa-393a04155f34 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:11.001700 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace20801-4dc5-4da1-b1b9-f673c17ad5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:11.002791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace20801-4dc5-4da1-b1b9-f673c17ad5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:11.003077 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace20801-4dc5-4da1-b1b9-f673c17ad5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:11.003356 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace20801-4dc5-4da1-b1b9-f673c17ad5f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:11.004136 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace20801-4dc5-4da1-b1b9-f673c17ad5f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:11.005891 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace20801-4dc5-4da1-b1b9-f673c17ad5f4 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:11.018221 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1428/2825] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:16:11.022085 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be134c09-01b9-400d-ae6c-e3bdddcde56f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:11.034576 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be134c09-01b9-400d-ae6c-e3bdddcde56f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:11.039995 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a196f-0da8-4a11-9f44-0f5fbce8de0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:11.041158 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a196f-0da8-4a11-9f44-0f5fbce8de0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:11.041622 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a196f-0da8-4a11-9f44-0f5fbce8de0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:11.042057 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a196f-0da8-4a11-9f44-0f5fbce8de0e admin admin] PATH_INFO: `/v3/projects/e9e45399f0594cf29d712bc99837c314/users/719a1807c5c244bab34c3357169a9539/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:11.061506 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a196f-0da8-4a11-9f44-0f5fbce8de0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9e45399f0594cf29d712bc99837c314, user_id=719a1807c5c244bab34c3357169a9539, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:11.064570 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a196f-0da8-4a11-9f44-0f5fbce8de0e admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:11.096072 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1398/2826] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:16:11 2022] PUT /identity/v3/projects/e9e45399f0594cf29d712bc99837c314/users/719a1807c5c244bab34c3357169a9539/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:16:11.097914 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d48c1-f8e6-406d-8a74-8e3fdf1b648f None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:11.098209 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d48c1-f8e6-406d-8a74-8e3fdf1b648f None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:11.098494 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d48c1-f8e6-406d-8a74-8e3fdf1b648f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:11.127452 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-617d48c1-f8e6-406d-8a74-8e3fdf1b648f None None] MFA Rules not processed for user `719a1807c5c244bab34c3357169a9539`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:11.128722 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-617d48c1-f8e6-406d-8a74-8e3fdf1b648f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:11.205988 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1429/2827] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:11.210473 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a196f-0da8-4a11-9f44-0f5fbce8de0e admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:11.275494 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a196f-0da8-4a11-9f44-0f5fbce8de0e admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:11.281302 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5bc27-c6e0-4cb7-ac79-662f5e77bce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:11.281966 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5bc27-c6e0-4cb7-ac79-662f5e77bce0 admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:11.282253 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5bc27-c6e0-4cb7-ac79-662f5e77bce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:11.282527 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5bc27-c6e0-4cb7-ac79-662f5e77bce0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:11.283206 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5bc27-c6e0-4cb7-ac79-662f5e77bce0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:11.285244 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5bc27-c6e0-4cb7-ac79-662f5e77bce0 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:11.299692 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:11.315408 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1399/2828] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:16:11 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:16:11.319595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d48c1-f8e6-406d-8a74-8e3fdf1b648f None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:11.332832 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d48c1-f8e6-406d-8a74-8e3fdf1b648f None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:11.392840 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f4b7e-566d-48e1-9e24-580efbcbc6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:11.393508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f4b7e-566d-48e1-9e24-580efbcbc6ac admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:11.393786 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f4b7e-566d-48e1-9e24-580efbcbc6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:11.394058 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f4b7e-566d-48e1-9e24-580efbcbc6ac admin admin] PATH_INFO: `/v3/users` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:11.394852 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f4b7e-566d-48e1-9e24-580efbcbc6ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:11.396808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f4b7e-566d-48e1-9e24-580efbcbc6ac admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:11.416920 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1430/2829] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:16:11 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:16:11.422344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5bc27-c6e0-4cb7-ac79-662f5e77bce0 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:11.433911 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5bc27-c6e0-4cb7-ac79-662f5e77bce0 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:11.439508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576a6ae-d74b-4226-aa9b-edcdda7cdf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:11.440302 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576a6ae-d74b-4226-aa9b-edcdda7cdf90 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:11.440619 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576a6ae-d74b-4226-aa9b-edcdda7cdf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:11.440902 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576a6ae-d74b-4226-aa9b-edcdda7cdf90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:11.441641 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576a6ae-d74b-4226-aa9b-edcdda7cdf90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:11.443256 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576a6ae-d74b-4226-aa9b-edcdda7cdf90 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:11.454455 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1400/2830] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:16:11.458923 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f4b7e-566d-48e1-9e24-580efbcbc6ac admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:11.469319 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f4b7e-566d-48e1-9e24-580efbcbc6ac admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:11.474870 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b41b8-4025-4150-ba29-0ed3525dc4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:11.475792 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b41b8-4025-4150-ba29-0ed3525dc4a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:11.476075 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b41b8-4025-4150-ba29-0ed3525dc4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:11.476350 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b41b8-4025-4150-ba29-0ed3525dc4a9 admin admin] PATH_INFO: `/v3/projects/a38134ce437c4c2ebec35e3061d685ab/users/806a1383052b427294e9fdbf42282307/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:11.493469 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b41b8-4025-4150-ba29-0ed3525dc4a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a38134ce437c4c2ebec35e3061d685ab, user_id=806a1383052b427294e9fdbf42282307, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:11.496138 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b41b8-4025-4150-ba29-0ed3525dc4a9 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:11.511731 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1431/2831] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:16:11 2022] PUT /identity/v3/projects/a38134ce437c4c2ebec35e3061d685ab/users/806a1383052b427294e9fdbf42282307/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:16:11.516371 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576a6ae-d74b-4226-aa9b-edcdda7cdf90 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:11.527022 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576a6ae-d74b-4226-aa9b-edcdda7cdf90 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:11.590461 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be8fd4-8601-4879-9d9e-84636e2a1528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:11.591138 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be8fd4-8601-4879-9d9e-84636e2a1528 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:11.591419 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be8fd4-8601-4879-9d9e-84636e2a1528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:11.591693 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be8fd4-8601-4879-9d9e-84636e2a1528 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:11.592575 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be8fd4-8601-4879-9d9e-84636e2a1528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:11.594200 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be8fd4-8601-4879-9d9e-84636e2a1528 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:11.605404 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1401/2832] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:16:11.610041 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b41b8-4025-4150-ba29-0ed3525dc4a9 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:11.621745 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b41b8-4025-4150-ba29-0ed3525dc4a9 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:11.627247 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530527e2-b717-427f-9edf-ce7c08ec8bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:11.628136 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530527e2-b717-427f-9edf-ce7c08ec8bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:11.628441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530527e2-b717-427f-9edf-ce7c08ec8bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:11.628725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530527e2-b717-427f-9edf-ce7c08ec8bd3 admin admin] PATH_INFO: `/v3/projects/a38134ce437c4c2ebec35e3061d685ab/users/806a1383052b427294e9fdbf42282307/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:11.630807 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530527e2-b717-427f-9edf-ce7c08ec8bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a38134ce437c4c2ebec35e3061d685ab, user_id=806a1383052b427294e9fdbf42282307, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:11.633563 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530527e2-b717-427f-9edf-ce7c08ec8bd3 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:11.644774 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1432/2833] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:16:11 2022] PUT /identity/v3/projects/a38134ce437c4c2ebec35e3061d685ab/users/806a1383052b427294e9fdbf42282307/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:11.656165 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2d36f-5a2c-4f35-af6f-b2f9b5ed7e4b None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:11.656487 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2d36f-5a2c-4f35-af6f-b2f9b5ed7e4b None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:11.656771 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2d36f-5a2c-4f35-af6f-b2f9b5ed7e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:11.683586 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-9ec2d36f-5a2c-4f35-af6f-b2f9b5ed7e4b None None] MFA Rules not processed for user `806a1383052b427294e9fdbf42282307`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:11.684834 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-9ec2d36f-5a2c-4f35-af6f-b2f9b5ed7e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:11.766797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1402/2834] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:11.780443 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530527e2-b717-427f-9edf-ce7c08ec8bd3 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:11.782289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-530527e2-b717-427f-9edf-ce7c08ec8bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:11.818023 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:11.844860 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530527e2-b717-427f-9edf-ce7c08ec8bd3 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:11.848784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805cf68-6da8-404c-b6ec-f32cf0825cba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:11.849439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805cf68-6da8-404c-b6ec-f32cf0825cba service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:11.849714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805cf68-6da8-404c-b6ec-f32cf0825cba service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:11.849986 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805cf68-6da8-404c-b6ec-f32cf0825cba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:11.859484 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805cf68-6da8-404c-b6ec-f32cf0825cba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:11.860958 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805cf68-6da8-404c-b6ec-f32cf0825cba service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:11.871538 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1433/2835] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:16:11 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:16:12.457772 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2d36f-5a2c-4f35-af6f-b2f9b5ed7e4b None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:12.521929 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2d36f-5a2c-4f35-af6f-b2f9b5ed7e4b None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:12.526767 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a4a13-c27d-430c-97e1-85e64c4fa127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:12.527633 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a4a13-c27d-430c-97e1-85e64c4fa127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:12.527907 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a4a13-c27d-430c-97e1-85e64c4fa127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:12.528172 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a4a13-c27d-430c-97e1-85e64c4fa127 admin admin] PATH_INFO: `/v3/users/719a1807c5c244bab34c3357169a9539` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:12.529408 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a4a13-c27d-430c-97e1-85e64c4fa127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=719a1807c5c244bab34c3357169a9539)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:12.531445 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a4a13-c27d-430c-97e1-85e64c4fa127 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:12.536086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:12.582543 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-063a4a13-c27d-430c-97e1-85e64c4fa127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '719a1807c5c244bab34c3357169a9539'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:12.583397 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-063a4a13-c27d-430c-97e1-85e64c4fa127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '719a1807c5c244bab34c3357169a9539'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:12.594880 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-063a4a13-c27d-430c-97e1-85e64c4fa127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '719a1807c5c244bab34c3357169a9539'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:12.603061 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-063a4a13-c27d-430c-97e1-85e64c4fa127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '719a1807c5c244bab34c3357169a9539'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:12.603848 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-063a4a13-c27d-430c-97e1-85e64c4fa127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '719a1807c5c244bab34c3357169a9539'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:12.622086 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1403/2836] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:16:12 2022] DELETE /identity/v3/users/719a1807c5c244bab34c3357169a9539 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:12.626764 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8ec57-8f8f-43bb-810f-026029a470f0 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:12.627246 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8ec57-8f8f-43bb-810f-026029a470f0 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:12.627361 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8ec57-8f8f-43bb-810f-026029a470f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:12.645718 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-2bb8ec57-8f8f-43bb-810f-026029a470f0 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:12.647056 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2bb8ec57-8f8f-43bb-810f-026029a470f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:12.710427 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1434/2837] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:12.720539 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a4a13-c27d-430c-97e1-85e64c4fa127 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:12.722106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-063a4a13-c27d-430c-97e1-85e64c4fa127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:12.786282 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a4a13-c27d-430c-97e1-85e64c4fa127 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:12.790108 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b361a6-fc78-4afa-94fb-e3c202e86cea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:12.790832 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b361a6-fc78-4afa-94fb-e3c202e86cea service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:12.791114 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b361a6-fc78-4afa-94fb-e3c202e86cea service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:12.791389 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b361a6-fc78-4afa-94fb-e3c202e86cea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:12.801592 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b361a6-fc78-4afa-94fb-e3c202e86cea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:12.803084 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b361a6-fc78-4afa-94fb-e3c202e86cea service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:12.813357 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1404/2838] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:16:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:16:13.101870 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8ec57-8f8f-43bb-810f-026029a470f0 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:13.103470 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-2bb8ec57-8f8f-43bb-810f-026029a470f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:13.116490 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8ec57-8f8f-43bb-810f-026029a470f0 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:13.123632 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97507ee8-e056-43d4-ba52-16ed0a4b252f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:13.124895 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97507ee8-e056-43d4-ba52-16ed0a4b252f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:13.125181 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97507ee8-e056-43d4-ba52-16ed0a4b252f admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:13.125455 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97507ee8-e056-43d4-ba52-16ed0a4b252f admin admin] PATH_INFO: `/v3/projects/e9e45399f0594cf29d712bc99837c314` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:13.126863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97507ee8-e056-43d4-ba52-16ed0a4b252f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9e45399f0594cf29d712bc99837c314)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:13.128874 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97507ee8-e056-43d4-ba52-16ed0a4b252f admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:13.185485 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-97507ee8-e056-43d4-ba52-16ed0a4b252f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9e45399f0594cf29d712bc99837c314', 'request_id': 'req-97507ee8-e056-43d4-ba52-16ed0a4b252f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:13.191902 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-97507ee8-e056-43d4-ba52-16ed0a4b252f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9e45399f0594cf29d712bc99837c314', 'request_id': 'req-97507ee8-e056-43d4-ba52-16ed0a4b252f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:13.199471 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-97507ee8-e056-43d4-ba52-16ed0a4b252f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9e45399f0594cf29d712bc99837c314', 'request_id': 'req-97507ee8-e056-43d4-ba52-16ed0a4b252f'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:13.213402 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-97507ee8-e056-43d4-ba52-16ed0a4b252f admin admin] The token cache is being invalidate because project e9e45399f0594cf29d712bc99837c314 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:16:13.213689 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-97507ee8-e056-43d4-ba52-16ed0a4b252f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:13.214300 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-97507ee8-e056-43d4-ba52-16ed0a4b252f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:13.215844 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1435/2839] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:16:13 2022] DELETE /identity/v3/projects/e9e45399f0594cf29d712bc99837c314 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:13.220378 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b361a6-fc78-4afa-94fb-e3c202e86cea service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:13.231683 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b361a6-fc78-4afa-94fb-e3c202e86cea service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:13.292870 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e0e79-0166-4f99-8a64-87ab3d91a81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:13.294158 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e0e79-0166-4f99-8a64-87ab3d91a81f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:13.294474 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e0e79-0166-4f99-8a64-87ab3d91a81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:13.294749 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e0e79-0166-4f99-8a64-87ab3d91a81f admin admin] PATH_INFO: `/v3/users/806a1383052b427294e9fdbf42282307` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:13.296007 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e0e79-0166-4f99-8a64-87ab3d91a81f admin admin] RBAC: Authorizing `identity:delete_user(user_id=806a1383052b427294e9fdbf42282307)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:13.298004 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e0e79-0166-4f99-8a64-87ab3d91a81f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:13.328629 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:13.350385 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-923e0e79-0166-4f99-8a64-87ab3d91a81f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '806a1383052b427294e9fdbf42282307'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:13.350935 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-923e0e79-0166-4f99-8a64-87ab3d91a81f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '806a1383052b427294e9fdbf42282307'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:13.359681 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-923e0e79-0166-4f99-8a64-87ab3d91a81f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '806a1383052b427294e9fdbf42282307'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:13.366936 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-923e0e79-0166-4f99-8a64-87ab3d91a81f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '806a1383052b427294e9fdbf42282307'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:13.367614 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-923e0e79-0166-4f99-8a64-87ab3d91a81f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '806a1383052b427294e9fdbf42282307'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:13.385719 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1405/2840] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:16:13 2022] DELETE /identity/v3/users/806a1383052b427294e9fdbf42282307 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:13.666610 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97507ee8-e056-43d4-ba52-16ed0a4b252f admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:13.668062 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-97507ee8-e056-43d4-ba52-16ed0a4b252f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:13.682508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97507ee8-e056-43d4-ba52-16ed0a4b252f admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:13.735660 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:13.742865 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42f11d-f6fa-46d9-85eb-4b33489a7ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:13.743791 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42f11d-f6fa-46d9-85eb-4b33489a7ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:13.744088 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42f11d-f6fa-46d9-85eb-4b33489a7ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:13.744377 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42f11d-f6fa-46d9-85eb-4b33489a7ad0 admin admin] PATH_INFO: `/v3/projects/a38134ce437c4c2ebec35e3061d685ab` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:13.745981 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42f11d-f6fa-46d9-85eb-4b33489a7ad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a38134ce437c4c2ebec35e3061d685ab)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:13.748104 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42f11d-f6fa-46d9-85eb-4b33489a7ad0 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:13.799179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7f42f11d-f6fa-46d9-85eb-4b33489a7ad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a38134ce437c4c2ebec35e3061d685ab', 'request_id': 'req-7f42f11d-f6fa-46d9-85eb-4b33489a7ad0'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:13.804998 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7f42f11d-f6fa-46d9-85eb-4b33489a7ad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a38134ce437c4c2ebec35e3061d685ab', 'request_id': 'req-7f42f11d-f6fa-46d9-85eb-4b33489a7ad0'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:13.811808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7f42f11d-f6fa-46d9-85eb-4b33489a7ad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a38134ce437c4c2ebec35e3061d685ab', 'request_id': 'req-7f42f11d-f6fa-46d9-85eb-4b33489a7ad0'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:13.826055 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7f42f11d-f6fa-46d9-85eb-4b33489a7ad0 admin admin] The token cache is being invalidate because project a38134ce437c4c2ebec35e3061d685ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:16:13.826340 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7f42f11d-f6fa-46d9-85eb-4b33489a7ad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:13.826982 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-7f42f11d-f6fa-46d9-85eb-4b33489a7ad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:13.828576 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1436/2841] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:16:13 2022] DELETE /identity/v3/projects/a38134ce437c4c2ebec35e3061d685ab => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:13.833643 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639c820-5e20-4739-8504-1f1ca034b4b5 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:13.834143 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639c820-5e20-4739-8504-1f1ca034b4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:13.834246 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639c820-5e20-4739-8504-1f1ca034b4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:13.852878 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-4639c820-5e20-4739-8504-1f1ca034b4b5 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:13.854055 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4639c820-5e20-4739-8504-1f1ca034b4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:13.919816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1406/2842] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:13.928032 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99cf04-6045-4158-89f4-435a47456dd9 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:13.928319 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99cf04-6045-4158-89f4-435a47456dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:13.928628 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99cf04-6045-4158-89f4-435a47456dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:13.947487 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-df99cf04-6045-4158-89f4-435a47456dd9 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:13.948714 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-df99cf04-6045-4158-89f4-435a47456dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:13.956906 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1437/2843] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:13.961353 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639c820-5e20-4739-8504-1f1ca034b4b5 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:13.971974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639c820-5e20-4739-8504-1f1ca034b4b5 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:13.977902 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898e5bb-9f6d-4188-bc8e-19a3166f627f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:13.978819 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898e5bb-9f6d-4188-bc8e-19a3166f627f admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:13.979106 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898e5bb-9f6d-4188-bc8e-19a3166f627f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:13.979385 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898e5bb-9f6d-4188-bc8e-19a3166f627f admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:13.980127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898e5bb-9f6d-4188-bc8e-19a3166f627f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:16:13.980465 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898e5bb-9f6d-4188-bc8e-19a3166f627f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:13.982082 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898e5bb-9f6d-4188-bc8e-19a3166f627f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:13.995084 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1407/2844] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:16:13.999504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df99cf04-6045-4158-89f4-435a47456dd9 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:14.009876 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df99cf04-6045-4158-89f4-435a47456dd9 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:14.016027 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08f963-ebd1-4ba3-bae3-7d31ac064894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:14.016721 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08f963-ebd1-4ba3-bae3-7d31ac064894 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:14.017011 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08f963-ebd1-4ba3-bae3-7d31ac064894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:14.017289 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08f963-ebd1-4ba3-bae3-7d31ac064894 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:14.017987 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08f963-ebd1-4ba3-bae3-7d31ac064894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:14.019936 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08f963-ebd1-4ba3-bae3-7d31ac064894 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:14.048708 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1438/2845] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:16:13 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:16:14.052824 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898e5bb-9f6d-4188-bc8e-19a3166f627f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:14.063271 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898e5bb-9f6d-4188-bc8e-19a3166f627f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:14.129947 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42ad3f-8ef4-4d28-aeb5-07e0d40e5b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:14.130740 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42ad3f-8ef4-4d28-aeb5-07e0d40e5b7f admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:14.131091 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42ad3f-8ef4-4d28-aeb5-07e0d40e5b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:14.131363 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42ad3f-8ef4-4d28-aeb5-07e0d40e5b7f admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:14.132452 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c42ad3f-8ef4-4d28-aeb5-07e0d40e5b7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:14.135775 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c42ad3f-8ef4-4d28-aeb5-07e0d40e5b7f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:14.160715 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1408/2846] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:16:14 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:16:14.164969 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08f963-ebd1-4ba3-bae3-7d31ac064894 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:14.175733 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08f963-ebd1-4ba3-bae3-7d31ac064894 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:14.182457 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2542e-84dc-4b7a-9548-9b1abcfe6334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:14.183140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2542e-84dc-4b7a-9548-9b1abcfe6334 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:14.183430 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2542e-84dc-4b7a-9548-9b1abcfe6334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:14.183705 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2542e-84dc-4b7a-9548-9b1abcfe6334 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:14.184528 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2542e-84dc-4b7a-9548-9b1abcfe6334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:14.186098 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2542e-84dc-4b7a-9548-9b1abcfe6334 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:14.198228 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1439/2847] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:16:14.202416 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42ad3f-8ef4-4d28-aeb5-07e0d40e5b7f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:14.213418 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42ad3f-8ef4-4d28-aeb5-07e0d40e5b7f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:14.219525 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6aa807-3296-461f-b97e-65ab9afdcd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:14.220496 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6aa807-3296-461f-b97e-65ab9afdcd24 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:14.220802 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6aa807-3296-461f-b97e-65ab9afdcd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:14.221103 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6aa807-3296-461f-b97e-65ab9afdcd24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:14.221801 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6aa807-3296-461f-b97e-65ab9afdcd24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:14.223475 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6aa807-3296-461f-b97e-65ab9afdcd24 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:14.234330 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1409/2848] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:16:14.238596 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2542e-84dc-4b7a-9548-9b1abcfe6334 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:14.249687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2542e-84dc-4b7a-9548-9b1abcfe6334 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:14.254387 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6f4bc-a8cd-48cf-9852-1136b0efc511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:14.255279 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6f4bc-a8cd-48cf-9852-1136b0efc511 admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:14.255565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6f4bc-a8cd-48cf-9852-1136b0efc511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:14.255844 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6f4bc-a8cd-48cf-9852-1136b0efc511 admin admin] PATH_INFO: `/v3/projects/0e4ea6b0bf9e4ba5b7ecf8dbf788959c/users/89ec7c6222324fc195c0157765f6111d/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:14.271315 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6f4bc-a8cd-48cf-9852-1136b0efc511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e4ea6b0bf9e4ba5b7ecf8dbf788959c, user_id=89ec7c6222324fc195c0157765f6111d, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:14.274177 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6f4bc-a8cd-48cf-9852-1136b0efc511 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:14.288954 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1440/2849] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:16:14 2022] PUT /identity/v3/projects/0e4ea6b0bf9e4ba5b7ecf8dbf788959c/users/89ec7c6222324fc195c0157765f6111d/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:14.302333 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e771d65-808d-4346-a45d-6624e5baa052 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:14.302642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e771d65-808d-4346-a45d-6624e5baa052 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:14.303022 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e771d65-808d-4346-a45d-6624e5baa052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:14.329230 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:14.337908 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-1e771d65-808d-4346-a45d-6624e5baa052 None None] MFA Rules not processed for user `89ec7c6222324fc195c0157765f6111d`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:14.339508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-1e771d65-808d-4346-a45d-6624e5baa052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:14.409570 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1410/2850] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:16:14.421254 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6f4bc-a8cd-48cf-9852-1136b0efc511 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:14.424321 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5ae6f4bc-a8cd-48cf-9852-1136b0efc511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:14.490508 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6f4bc-a8cd-48cf-9852-1136b0efc511 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:14.494599 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7625e0-cbb3-4c8c-b4d8-699577a79a47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b00152c72a1f4755b09c1dbd1d874b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:14.495262 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7625e0-cbb3-4c8c-b4d8-699577a79a47 service cinder] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:14.495541 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7625e0-cbb3-4c8c-b4d8-699577a79a47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:14.495818 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7625e0-cbb3-4c8c-b4d8-699577a79a47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:14.504862 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7625e0-cbb3-4c8c-b4d8-699577a79a47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:14.506332 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7625e0-cbb3-4c8c-b4d8-699577a79a47 service cinder] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:14.516043 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1441/2851] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:16:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:16:24.406480 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=27 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:24.511892 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:30.887301 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e771d65-808d-4346-a45d-6624e5baa052 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:30.958561 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e771d65-808d-4346-a45d-6624e5baa052 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:30.963539 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd2d6f-b8ca-42a3-8248-bfe4082a7595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:30.964490 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd2d6f-b8ca-42a3-8248-bfe4082a7595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:30.964752 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd2d6f-b8ca-42a3-8248-bfe4082a7595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:30.965021 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd2d6f-b8ca-42a3-8248-bfe4082a7595 admin admin] PATH_INFO: `/v3/users/89ec7c6222324fc195c0157765f6111d` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:30.966286 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd2d6f-b8ca-42a3-8248-bfe4082a7595 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89ec7c6222324fc195c0157765f6111d)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:30.968210 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd2d6f-b8ca-42a3-8248-bfe4082a7595 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:31.017675 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ebfd2d6f-b8ca-42a3-8248-bfe4082a7595 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89ec7c6222324fc195c0157765f6111d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:31.018386 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ebfd2d6f-b8ca-42a3-8248-bfe4082a7595 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89ec7c6222324fc195c0157765f6111d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:31.027530 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ebfd2d6f-b8ca-42a3-8248-bfe4082a7595 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89ec7c6222324fc195c0157765f6111d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:31.034434 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ebfd2d6f-b8ca-42a3-8248-bfe4082a7595 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89ec7c6222324fc195c0157765f6111d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:31.035055 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-ebfd2d6f-b8ca-42a3-8248-bfe4082a7595 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89ec7c6222324fc195c0157765f6111d'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:31.052690 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1411/2852] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:16:30 2022] DELETE /identity/v3/users/89ec7c6222324fc195c0157765f6111d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:31.058435 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8939cf-f4c8-49d0-b7d6-5743dab379a4 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:31.058926 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8939cf-f4c8-49d0-b7d6-5743dab379a4 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:31.059025 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8939cf-f4c8-49d0-b7d6-5743dab379a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:31.080797 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-5e8939cf-f4c8-49d0-b7d6-5743dab379a4 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:31.082008 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5e8939cf-f4c8-49d0-b7d6-5743dab379a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:31.143687 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1442/2853] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:31.153986 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd2d6f-b8ca-42a3-8248-bfe4082a7595 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:31.155552 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ebfd2d6f-b8ca-42a3-8248-bfe4082a7595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:31.215103 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd2d6f-b8ca-42a3-8248-bfe4082a7595 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:31.219099 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680c780-9751-44fc-b721-e0482c2ee9b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:31.219742 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680c780-9751-44fc-b721-e0482c2ee9b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:31.220026 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680c780-9751-44fc-b721-e0482c2ee9b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:31.220302 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680c780-9751-44fc-b721-e0482c2ee9b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:31.230082 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680c780-9751-44fc-b721-e0482c2ee9b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:31.231519 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680c780-9751-44fc-b721-e0482c2ee9b5 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:31.241208 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1412/2854] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:16:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:16:31.533894 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8939cf-f4c8-49d0-b7d6-5743dab379a4 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:31.535584 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-5e8939cf-f4c8-49d0-b7d6-5743dab379a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:31.551176 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8939cf-f4c8-49d0-b7d6-5743dab379a4 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:31.558383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c65ad-0905-47fa-ac85-7f9d99eab07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:31.559362 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c65ad-0905-47fa-ac85-7f9d99eab07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:31.559630 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c65ad-0905-47fa-ac85-7f9d99eab07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:31.559751 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c65ad-0905-47fa-ac85-7f9d99eab07e admin admin] PATH_INFO: `/v3/projects/0e4ea6b0bf9e4ba5b7ecf8dbf788959c` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:31.561095 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c65ad-0905-47fa-ac85-7f9d99eab07e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e4ea6b0bf9e4ba5b7ecf8dbf788959c)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:31.563014 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c65ad-0905-47fa-ac85-7f9d99eab07e admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:31.612070 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fb8c65ad-0905-47fa-ac85-7f9d99eab07e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e4ea6b0bf9e4ba5b7ecf8dbf788959c', 'request_id': 'req-fb8c65ad-0905-47fa-ac85-7f9d99eab07e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:31.618252 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fb8c65ad-0905-47fa-ac85-7f9d99eab07e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e4ea6b0bf9e4ba5b7ecf8dbf788959c', 'request_id': 'req-fb8c65ad-0905-47fa-ac85-7f9d99eab07e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:31.624550 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fb8c65ad-0905-47fa-ac85-7f9d99eab07e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e4ea6b0bf9e4ba5b7ecf8dbf788959c', 'request_id': 'req-fb8c65ad-0905-47fa-ac85-7f9d99eab07e'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:31.639981 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fb8c65ad-0905-47fa-ac85-7f9d99eab07e admin admin] The token cache is being invalidate because project 0e4ea6b0bf9e4ba5b7ecf8dbf788959c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:16:31.640268 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fb8c65ad-0905-47fa-ac85-7f9d99eab07e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:31.640941 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-fb8c65ad-0905-47fa-ac85-7f9d99eab07e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:16:31.642498 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1443/2855] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:16:31 2022] DELETE /identity/v3/projects/0e4ea6b0bf9e4ba5b7ecf8dbf788959c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:31.649471 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e5b5f-d490-4797-93ee-e2d51e770812 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:31.649703 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e5b5f-d490-4797-93ee-e2d51e770812 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:31.649983 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e5b5f-d490-4797-93ee-e2d51e770812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:31.669733 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-7e9e5b5f-d490-4797-93ee-e2d51e770812 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:31.670950 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-7e9e5b5f-d490-4797-93ee-e2d51e770812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:31.690231 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:31.736935 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1413/2856] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:31.747516 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3657dd-a0f6-4a82-910b-33ddc26919d6 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:31.747839 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3657dd-a0f6-4a82-910b-33ddc26919d6 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:31.748083 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3657dd-a0f6-4a82-910b-33ddc26919d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:31.767864 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-1b3657dd-a0f6-4a82-910b-33ddc26919d6 None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:31.769112 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1b3657dd-a0f6-4a82-910b-33ddc26919d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:31.778857 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1444/2857] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:31.783300 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e5b5f-d490-4797-93ee-e2d51e770812 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:31.793851 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e5b5f-d490-4797-93ee-e2d51e770812 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:31.799508 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778f1d5-a0b2-4a65-9a43-91e203228257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:31.800185 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778f1d5-a0b2-4a65-9a43-91e203228257 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:31.800501 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778f1d5-a0b2-4a65-9a43-91e203228257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:31.800781 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778f1d5-a0b2-4a65-9a43-91e203228257 admin admin] PATH_INFO: `/v3/domains` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:31.801510 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778f1d5-a0b2-4a65-9a43-91e203228257 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80125) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 25 16:16:31.801814 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778f1d5-a0b2-4a65-9a43-91e203228257 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:31.803414 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778f1d5-a0b2-4a65-9a43-91e203228257 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:31.815718 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1414/2858] 10.222.0.13 () {58 vars in 1152 bytes} [Sat Jun 25 16:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 25 16:16:31.820018 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3657dd-a0f6-4a82-910b-33ddc26919d6 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:31.829650 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3657dd-a0f6-4a82-910b-33ddc26919d6 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:31.834836 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab783e-07bb-4cf6-8184-5ee1bb4bf5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:31.835478 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab783e-07bb-4cf6-8184-5ee1bb4bf5a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:31.835766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab783e-07bb-4cf6-8184-5ee1bb4bf5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:31.836402 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab783e-07bb-4cf6-8184-5ee1bb4bf5a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:31.836960 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab783e-07bb-4cf6-8184-5ee1bb4bf5a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:31.838882 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab783e-07bb-4cf6-8184-5ee1bb4bf5a4 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:31.866962 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1445/2859] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:16:31 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 25 16:16:31.872662 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778f1d5-a0b2-4a65-9a43-91e203228257 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:31.887855 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778f1d5-a0b2-4a65-9a43-91e203228257 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:31.950833 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63c70a-bae8-4427-b37b-b56967694c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:31.951901 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63c70a-bae8-4427-b37b-b56967694c3f admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:31.952361 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63c70a-bae8-4427-b37b-b56967694c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:31.952878 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63c70a-bae8-4427-b37b-b56967694c3f admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:31.954000 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63c70a-bae8-4427-b37b-b56967694c3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:31.957211 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63c70a-bae8-4427-b37b-b56967694c3f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:31.980977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1415/2860] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:16:31 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:16:31.985136 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab783e-07bb-4cf6-8184-5ee1bb4bf5a4 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:31.995925 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab783e-07bb-4cf6-8184-5ee1bb4bf5a4 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:32.000850 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdab811-f871-4291-8207-448508da5a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:32.001513 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdab811-f871-4291-8207-448508da5a7b admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:32.001797 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdab811-f871-4291-8207-448508da5a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:32.002073 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdab811-f871-4291-8207-448508da5a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:32.002800 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdab811-f871-4291-8207-448508da5a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:32.004442 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdab811-f871-4291-8207-448508da5a7b admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:32.015460 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1446/2861] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:16:32.019622 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63c70a-bae8-4427-b37b-b56967694c3f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:32.030730 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63c70a-bae8-4427-b37b-b56967694c3f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:32.036583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3722bc-71ee-4e2b-b44b-12a9de445fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:32.037236 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3722bc-71ee-4e2b-b44b-12a9de445fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:32.037513 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3722bc-71ee-4e2b-b44b-12a9de445fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:32.037780 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3722bc-71ee-4e2b-b44b-12a9de445fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:32.038516 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3722bc-71ee-4e2b-b44b-12a9de445fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:32.040168 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3722bc-71ee-4e2b-b44b-12a9de445fb0 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:32.050617 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1416/2862] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:16:32.054803 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdab811-f871-4291-8207-448508da5a7b admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:32.065488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdab811-f871-4291-8207-448508da5a7b admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:32.071052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674601f-e719-4172-a903-bc05133e8e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:32.071928 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674601f-e719-4172-a903-bc05133e8e3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:32.072212 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674601f-e719-4172-a903-bc05133e8e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:32.072509 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674601f-e719-4172-a903-bc05133e8e3c admin admin] PATH_INFO: `/v3/projects/b1730bdd7de14144bf75c92004923f9a/users/c7b128ef63b94a71b7377a08aae7990f/roles/8b9bae8f81854757b4f389d3bd22f595` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:32.087741 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674601f-e719-4172-a903-bc05133e8e3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1730bdd7de14144bf75c92004923f9a, user_id=c7b128ef63b94a71b7377a08aae7990f, role_id=8b9bae8f81854757b4f389d3bd22f595)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:32.090481 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674601f-e719-4172-a903-bc05133e8e3c admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:32.105718 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1447/2863] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:16:32 2022] PUT /identity/v3/projects/b1730bdd7de14144bf75c92004923f9a/users/c7b128ef63b94a71b7377a08aae7990f/roles/8b9bae8f81854757b4f389d3bd22f595 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:32.111737 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f858d-4d85-4d2d-845b-e646ec5aac6e None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:32.112031 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f858d-4d85-4d2d-845b-e646ec5aac6e None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:32.112294 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f858d-4d85-4d2d-845b-e646ec5aac6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:32.132350 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a37f858d-4d85-4d2d-845b-e646ec5aac6e None None] MFA Rules not processed for user `c132d4e24cfd4bd197338f840d4d9eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:32.133838 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a37f858d-4d85-4d2d-845b-e646ec5aac6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:32.205783 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1417/2864] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:32.215608 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674601f-e719-4172-a903-bc05133e8e3c admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:32.217437 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-6674601f-e719-4172-a903-bc05133e8e3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:32.240852 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:32.278220 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674601f-e719-4172-a903-bc05133e8e3c admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:32.282463 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad3f1e-3db5-4abb-8db5-3db923160ee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:32.283103 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad3f1e-3db5-4abb-8db5-3db923160ee2 service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:32.283383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad3f1e-3db5-4abb-8db5-3db923160ee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:32.283659 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad3f1e-3db5-4abb-8db5-3db923160ee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:32.293152 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad3f1e-3db5-4abb-8db5-3db923160ee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:32.294566 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad3f1e-3db5-4abb-8db5-3db923160ee2 service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:32.304002 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1448/2865] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:16:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:16:36.480074 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b73b90-7ff4-4b29-b44f-0fe4998b2df5 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:36.480635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b73b90-7ff4-4b29-b44f-0fe4998b2df5 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:36.480635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b73b90-7ff4-4b29-b44f-0fe4998b2df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:36.511972 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-a3b73b90-7ff4-4b29-b44f-0fe4998b2df5 None None] MFA Rules not processed for user `c7b128ef63b94a71b7377a08aae7990f`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:36.513370 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-a3b73b90-7ff4-4b29-b44f-0fe4998b2df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:36.532239 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:36.596756 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1418/2866] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:36.597819 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad3f1e-3db5-4abb-8db5-3db923160ee2 service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:36.611386 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad3f1e-3db5-4abb-8db5-3db923160ee2 service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:36.616302 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506991ea-3ef4-4b4c-a66a-9eaa09120bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:36.617059 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506991ea-3ef4-4b4c-a66a-9eaa09120bed admin admin] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:36.617374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506991ea-3ef4-4b4c-a66a-9eaa09120bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:36.617657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506991ea-3ef4-4b4c-a66a-9eaa09120bed admin admin] PATH_INFO: `/v3/projects` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:36.618429 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506991ea-3ef4-4b4c-a66a-9eaa09120bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:36.620382 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506991ea-3ef4-4b4c-a66a-9eaa09120bed admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:36.649916 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1449/2867] 10.222.0.13 () {60 vars in 1152 bytes} [Sat Jun 25 16:16:36 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:16:36.654127 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b73b90-7ff4-4b29-b44f-0fe4998b2df5 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:36.666771 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b73b90-7ff4-4b29-b44f-0fe4998b2df5 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:36.731796 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2987f9-834f-4956-850e-a66723d4be3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:36.732630 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2987f9-834f-4956-850e-a66723d4be3d admin admin] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:36.732962 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2987f9-834f-4956-850e-a66723d4be3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:36.733326 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2987f9-834f-4956-850e-a66723d4be3d admin admin] PATH_INFO: `/v3/users` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:36.734598 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2987f9-834f-4956-850e-a66723d4be3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:36.736840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2987f9-834f-4956-850e-a66723d4be3d admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:36.765231 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1419/2868] 10.222.0.13 () {60 vars in 1143 bytes} [Sat Jun 25 16:16:36 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 25 16:16:36.766253 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506991ea-3ef4-4b4c-a66a-9eaa09120bed admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:36.778017 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506991ea-3ef4-4b4c-a66a-9eaa09120bed admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:36.783063 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c7119-6579-41c4-8643-1da7de09e498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:36.783950 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c7119-6579-41c4-8643-1da7de09e498 admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:36.784232 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c7119-6579-41c4-8643-1da7de09e498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:36.784544 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c7119-6579-41c4-8643-1da7de09e498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:36.785284 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c7119-6579-41c4-8643-1da7de09e498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:36.787243 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c7119-6579-41c4-8643-1da7de09e498 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:36.800315 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1450/2869] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 25 16:16:36.804323 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2987f9-834f-4956-850e-a66723d4be3d admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:36.817958 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2987f9-834f-4956-850e-a66723d4be3d admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:36.823421 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d99486-9a07-4664-89b2-8d24a5eaffb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:36.824324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d99486-9a07-4664-89b2-8d24a5eaffb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:36.824639 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d99486-9a07-4664-89b2-8d24a5eaffb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:36.824917 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d99486-9a07-4664-89b2-8d24a5eaffb0 admin admin] PATH_INFO: `/v3/projects/30f6f1f26604417994fee495603b5838/users/0284f5f052214c119531abb71c20dc90/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:36.842708 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d99486-9a07-4664-89b2-8d24a5eaffb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f6f1f26604417994fee495603b5838, user_id=0284f5f052214c119531abb71c20dc90, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:36.845420 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d99486-9a07-4664-89b2-8d24a5eaffb0 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:36.865070 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1420/2870] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:16:36 2022] PUT /identity/v3/projects/30f6f1f26604417994fee495603b5838/users/0284f5f052214c119531abb71c20dc90/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 25 16:16:36.866506 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c7119-6579-41c4-8643-1da7de09e498 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:36.877648 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c7119-6579-41c4-8643-1da7de09e498 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:36.942409 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81774e-05b8-468e-8447-c151c01a80fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:36.943092 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81774e-05b8-468e-8447-c151c01a80fe admin admin] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:36.943369 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81774e-05b8-468e-8447-c151c01a80fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:36.943641 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81774e-05b8-468e-8447-c151c01a80fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:36.944374 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81774e-05b8-468e-8447-c151c01a80fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:36.946094 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81774e-05b8-468e-8447-c151c01a80fe admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:36.960028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1451/2871] 10.222.0.13 () {58 vars in 1121 bytes} [Sat Jun 25 16:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 25 16:16:36.961259 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d99486-9a07-4664-89b2-8d24a5eaffb0 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:36.977956 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d99486-9a07-4664-89b2-8d24a5eaffb0 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:36.982991 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9c6b6-1ab4-4f84-b555-db0207335b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:36.983888 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9c6b6-1ab4-4f84-b555-db0207335b62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:36.984192 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9c6b6-1ab4-4f84-b555-db0207335b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:36.984520 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9c6b6-1ab4-4f84-b555-db0207335b62 admin admin] PATH_INFO: `/v3/projects/30f6f1f26604417994fee495603b5838/users/0284f5f052214c119531abb71c20dc90/roles/b1e13c449fa64b3e88b6832bb767214b` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:36.986485 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9c6b6-1ab4-4f84-b555-db0207335b62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f6f1f26604417994fee495603b5838, user_id=0284f5f052214c119531abb71c20dc90, role_id=b1e13c449fa64b3e88b6832bb767214b)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:36.989136 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9c6b6-1ab4-4f84-b555-db0207335b62 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:37.002888 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1421/2872] 10.222.0.13 () {60 vars in 1482 bytes} [Sat Jun 25 16:16:36 2022] PUT /identity/v3/projects/30f6f1f26604417994fee495603b5838/users/0284f5f052214c119531abb71c20dc90/roles/b1e13c449fa64b3e88b6832bb767214b => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:16:41.451296 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a289f-c8d1-43b5-9aa1-90a4852e0f31 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:41.451838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a289f-c8d1-43b5-9aa1-90a4852e0f31 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:41.451838 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a289f-c8d1-43b5-9aa1-90a4852e0f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:41.492080 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-a00a289f-c8d1-43b5-9aa1-90a4852e0f31 None None] MFA Rules not processed for user `0284f5f052214c119531abb71c20dc90`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:41.493346 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a00a289f-c8d1-43b5-9aa1-90a4852e0f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:41.588097 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1452/2873] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:41.601814 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9c6b6-1ab4-4f84-b555-db0207335b62 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:41.604395 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-19f9c6b6-1ab4-4f84-b555-db0207335b62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:41.709732 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9c6b6-1ab4-4f84-b555-db0207335b62 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:41.713977 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4f732-14d0-4bee-ad3c-c034020192ee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5cba086728824ec68fb0ee90fb3f99dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:41.714637 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4f732-14d0-4bee-ad3c-c034020192ee service glance] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:41.714920 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4f732-14d0-4bee-ad3c-c034020192ee service glance] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:41.715224 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4f732-14d0-4bee-ad3c-c034020192ee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:41.747432 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:41.797562 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4f732-14d0-4bee-ad3c-c034020192ee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:41.799031 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4f732-14d0-4bee-ad3c-c034020192ee service glance] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:41.810297 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1422/2874] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:16:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:16:41.936705 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bf709-50a9-4c32-95f4-c454718c1d65 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:41.937013 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bf709-50a9-4c32-95f4-c454718c1d65 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:41.937782 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bf709-50a9-4c32-95f4-c454718c1d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:41.957133 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-de2bf709-50a9-4c32-95f4-c454718c1d65 None None] MFA Rules not processed for user `28b3e73439e64050b406a9f3e9b0160a`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:41.958523 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-de2bf709-50a9-4c32-95f4-c454718c1d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:41.965140 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:16:42.034495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1453/2875] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Jun 25 16:16:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 25 16:16:42.085928 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4f732-14d0-4bee-ad3c-c034020192ee service glance] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:42.087524 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-42f4f732-14d0-4bee-ad3c-c034020192ee service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:42.163395 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4f732-14d0-4bee-ad3c-c034020192ee service glance] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:42.169050 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4993928-769c-4549-af21-84a012b59e54 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19788c4239ca44f9a31a7d32086c8947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:42.169698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4993928-769c-4549-af21-84a012b59e54 service swift] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:42.169976 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4993928-769c-4549-af21-84a012b59e54 service swift] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:42.170243 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4993928-769c-4549-af21-84a012b59e54 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:42.179186 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4993928-769c-4549-af21-84a012b59e54 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:42.180794 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4993928-769c-4549-af21-84a012b59e54 service swift] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:42.189330 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1423/2876] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Jun 25 16:16:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 25 16:16:50.546488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e0a66-8e33-4a62-869b-f1a3868518a6 None None] REQUEST_METHOD: `POST` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:50.547299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e0a66-8e33-4a62-869b-f1a3868518a6 None None] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:50.547299 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e0a66-8e33-4a62-869b-f1a3868518a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:50.574483 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.auth.core [None req-219e0a66-8e33-4a62-869b-f1a3868518a6 None None] MFA Rules not processed for user `c7b128ef63b94a71b7377a08aae7990f`. Rule list: `[]` (Enabled: `True`). {{(pid=80124) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:50.575685 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-219e0a66-8e33-4a62-869b-f1a3868518a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:50.583862 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1454/2877] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:16:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:50.876583 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4993928-769c-4549-af21-84a012b59e54 service swift] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:50.878324 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c4993928-769c-4549-af21-84a012b59e54 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:50.944698 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4993928-769c-4549-af21-84a012b59e54 service swift] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:50.950203 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dacd3b-20a3-4861-9262-1c9e562dc639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:50.951015 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dacd3b-20a3-4861-9262-1c9e562dc639 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:50.951278 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dacd3b-20a3-4861-9262-1c9e562dc639 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:50.951548 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dacd3b-20a3-4861-9262-1c9e562dc639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:50.954223 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-44dacd3b-20a3-4861-9262-1c9e562dc639 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:50.966946 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dacd3b-20a3-4861-9262-1c9e562dc639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:50.968401 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dacd3b-20a3-4861-9262-1c9e562dc639 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:50.978494 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1424/2878] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:16:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:16:51.333538 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e0a66-8e33-4a62-869b-f1a3868518a6 None None] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:51.335171 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-219e0a66-8e33-4a62-869b-f1a3868518a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:51.421678 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e0a66-8e33-4a62-869b-f1a3868518a6 None None] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:51.427148 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67b580-3f5f-4137-bea5-faad0be0c1a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474c468a610140f8a5b18d5a1ed1d18e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:51.428174 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67b580-3f5f-4137-bea5-faad0be0c1a0 service placement] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:51.428503 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67b580-3f5f-4137-bea5-faad0be0c1a0 service placement] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:51.428780 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67b580-3f5f-4137-bea5-faad0be0c1a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:51.430845 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-3f67b580-3f5f-4137-bea5-faad0be0c1a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:51.443495 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67b580-3f5f-4137-bea5-faad0be0c1a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:51.445143 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67b580-3f5f-4137-bea5-faad0be0c1a0 service placement] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:51.455516 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1455/2879] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:16:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:16:52.739923 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac18b57-8cdc-49a1-b322-0c52e61d9ea6 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:52.740481 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac18b57-8cdc-49a1-b322-0c52e61d9ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:52.740878 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac18b57-8cdc-49a1-b322-0c52e61d9ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:52.759869 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-0ac18b57-8cdc-49a1-b322-0c52e61d9ea6 None None] MFA Rules not processed for user `28b3e73439e64050b406a9f3e9b0160a`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:16:52.761125 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-0ac18b57-8cdc-49a1-b322-0c52e61d9ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:16:52.768733 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1425/2880] 10.222.0.13 () {58 vars in 1009 bytes} [Sat Jun 25 16:16:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:16:52.782436 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67b580-3f5f-4137-bea5-faad0be0c1a0 service placement] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:16:52.794231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67b580-3f5f-4137-bea5-faad0be0c1a0 service placement] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:16:52.799198 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014a18c-bfd4-4572-ac34-602f2c472f01 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19788c4239ca44f9a31a7d32086c8947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:16:52.799899 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014a18c-bfd4-4572-ac34-602f2c472f01 service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:16:52.800179 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014a18c-bfd4-4572-ac34-602f2c472f01 service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:16:52.800441 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014a18c-bfd4-4572-ac34-602f2c472f01 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:16:52.809897 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014a18c-bfd4-4572-ac34-602f2c472f01 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:16:52.811647 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014a18c-bfd4-4572-ac34-602f2c472f01 service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:16:52.820554 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1456/2881] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Jun 25 16:16:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 25 16:17:00.954382 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac18b57-8cdc-49a1-b322-0c52e61d9ea6 None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:17:00.964007 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac18b57-8cdc-49a1-b322-0c52e61d9ea6 None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:17:00.967714 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17089c-7c8b-473d-a3d7-fb9f7da15394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:17:00.968350 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17089c-7c8b-473d-a3d7-fb9f7da15394 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:17:00.968671 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17089c-7c8b-473d-a3d7-fb9f7da15394 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:17:00.968938 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17089c-7c8b-473d-a3d7-fb9f7da15394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:17:00.970707 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-ac17089c-7c8b-473d-a3d7-fb9f7da15394 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:17:00.980494 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17089c-7c8b-473d-a3d7-fb9f7da15394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:17:00.981915 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17089c-7c8b-473d-a3d7-fb9f7da15394 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:17:00.990383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1426/2882] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:17:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 25 16:17:01.637676 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014a18c-bfd4-4572-ac34-602f2c472f01 service swift] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:17:01.639645 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-7014a18c-bfd4-4572-ac34-602f2c472f01 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:17:01.781497 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014a18c-bfd4-4572-ac34-602f2c472f01 service swift] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:17:01.788504 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ff378-1602-4754-8d94-cf3020279e51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:17:01.791363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ff378-1602-4754-8d94-cf3020279e51 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:17:01.791363 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ff378-1602-4754-8d94-cf3020279e51 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:17:01.791883 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ff378-1602-4754-8d94-cf3020279e51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:17:01.794033 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c08ff378-1602-4754-8d94-cf3020279e51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:17:01.808254 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ff378-1602-4754-8d94-cf3020279e51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:17:01.809921 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ff378-1602-4754-8d94-cf3020279e51 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:17:01.825863 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1457/2883] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:17:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:17:10.986609 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=83 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:17:11.819330 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=95 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:17:18.413571 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17089c-7c8b-473d-a3d7-fb9f7da15394 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:17:18.422656 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17089c-7c8b-473d-a3d7-fb9f7da15394 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:17:18.427193 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfda44-254f-426e-ab44-d26dafdfe098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:17:18.427842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfda44-254f-426e-ab44-d26dafdfe098 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:17:18.428119 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfda44-254f-426e-ab44-d26dafdfe098 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:17:18.428388 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfda44-254f-426e-ab44-d26dafdfe098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:17:18.438838 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfda44-254f-426e-ab44-d26dafdfe098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:17:18.440463 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfda44-254f-426e-ab44-d26dafdfe098 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:17:18.450441 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1427/2884] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:17:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:17:20.283520 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ff378-1602-4754-8d94-cf3020279e51 service nova] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:17:20.296695 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ff378-1602-4754-8d94-cf3020279e51 service nova] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:17:20.303862 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d8ea1-1919-41aa-8b95-3b05b82b1fb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:17:20.305035 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d8ea1-1919-41aa-8b95-3b05b82b1fb9 service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:17:20.305318 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d8ea1-1919-41aa-8b95-3b05b82b1fb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:17:20.305593 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d8ea1-1919-41aa-8b95-3b05b82b1fb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:17:20.307387 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-979d8ea1-1919-41aa-8b95-3b05b82b1fb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:17:20.320649 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d8ea1-1919-41aa-8b95-3b05b82b1fb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:17:20.322261 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d8ea1-1919-41aa-8b95-3b05b82b1fb9 service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:17:20.335724 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1458/2885] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:17:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:17:28.446115 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=6 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:17:30.330063 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=6 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:17:51.652861 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937dbf6-7c4e-44a6-ac60-bfa4107c6e24 None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:17:51.653407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937dbf6-7c4e-44a6-ac60-bfa4107c6e24 None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:17:51.653407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937dbf6-7c4e-44a6-ac60-bfa4107c6e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:17:51.672275 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-8937dbf6-7c4e-44a6-ac60-bfa4107c6e24 None None] MFA Rules not processed for user `0ef599055aed42f3a97fb30bba553fb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:17:51.673532 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-8937dbf6-7c4e-44a6-ac60-bfa4107c6e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:17:51.734948 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1428/2886] 10.222.0.13 () {58 vars in 957 bytes} [Sat Jun 25 16:17:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:18:01.726786 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=25 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:18:05.336611 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d8ea1-1919-41aa-8b95-3b05b82b1fb9 service nova] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:18:05.338048 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-979d8ea1-1919-41aa-8b95-3b05b82b1fb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:18:05.354461 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d8ea1-1919-41aa-8b95-3b05b82b1fb9 service nova] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:18:05.360532 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea6f00-c9eb-49fc-b18e-5a898a8fa61f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3622c5ccb7534e7c8e8342b4f6643bc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:18:05.361191 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea6f00-c9eb-49fc-b18e-5a898a8fa61f service nova] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:18:05.361473 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea6f00-c9eb-49fc-b18e-5a898a8fa61f service nova] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:18:05.361748 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea6f00-c9eb-49fc-b18e-5a898a8fa61f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:18:05.363652 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-c9ea6f00-c9eb-49fc-b18e-5a898a8fa61f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:18:05.377443 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea6f00-c9eb-49fc-b18e-5a898a8fa61f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:18:05.378617 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea6f00-c9eb-49fc-b18e-5a898a8fa61f service nova] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:18:05.390439 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1459/2887] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:18:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:18:15.385746 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=6 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:18:41.645313 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f80353-5d35-410e-b70d-c6725181bafa None None] REQUEST_METHOD: `POST` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:18:41.646018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f80353-5d35-410e-b70d-c6725181bafa None None] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:18:41.646018 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f80353-5d35-410e-b70d-c6725181bafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:18:41.664921 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.auth.core [None req-e2f80353-5d35-410e-b70d-c6725181bafa None None] MFA Rules not processed for user `28b3e73439e64050b406a9f3e9b0160a`. Rule list: `[]` (Enabled: `True`). {{(pid=80125) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 25 16:18:41.666126 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e2f80353-5d35-410e-b70d-c6725181bafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:18:41.674797 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1429/2888] 10.222.0.13 () {58 vars in 967 bytes} [Sat Jun 25 16:18:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 25 16:18:41.717267 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea6f00-c9eb-49fc-b18e-5a898a8fa61f service nova] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:18:41.729284 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea6f00-c9eb-49fc-b18e-5a898a8fa61f service nova] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:18:41.735656 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e9aaf-b577-45cf-a173-063ad1c093ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19788c4239ca44f9a31a7d32086c8947', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:18:41.736307 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e9aaf-b577-45cf-a173-063ad1c093ec service swift] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:18:41.736657 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e9aaf-b577-45cf-a173-063ad1c093ec service swift] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:18:41.736931 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e9aaf-b577-45cf-a173-063ad1c093ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:18:41.746126 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e9aaf-b577-45cf-a173-063ad1c093ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:18:41.747743 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e9aaf-b577-45cf-a173-063ad1c093ec service swift] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:18:41.756622 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1460/2889] 10.222.0.13 () {60 vars in 1384 bytes} [Sat Jun 25 16:18:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 25 16:18:45.181347 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f80353-5d35-410e-b70d-c6725181bafa None None] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:18:45.182814 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-e2f80353-5d35-410e-b70d-c6725181bafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:18:45.198906 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f80353-5d35-410e-b70d-c6725181bafa None None] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:18:45.204974 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7be5c-17c0-488e-97bf-804f66caba3b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474c468a610140f8a5b18d5a1ed1d18e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:18:45.205629 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7be5c-17c0-488e-97bf-804f66caba3b service placement] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:18:45.205926 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7be5c-17c0-488e-97bf-804f66caba3b service placement] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:18:45.206178 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7be5c-17c0-488e-97bf-804f66caba3b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:18:45.208320 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-b7e7be5c-17c0-488e-97bf-804f66caba3b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:18:45.220349 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e7be5c-17c0-488e-97bf-804f66caba3b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:18:45.222053 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e7be5c-17c0-488e-97bf-804f66caba3b service placement] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:18:45.234383 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1430/2890] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:18:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:18:48.349200 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e9aaf-b577-45cf-a173-063ad1c093ec service swift] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:18:48.360942 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e9aaf-b577-45cf-a173-063ad1c093ec service swift] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:18:48.367046 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5253823-31ba-4304-80a6-99012c7ce14b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474c468a610140f8a5b18d5a1ed1d18e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:18:48.367729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5253823-31ba-4304-80a6-99012c7ce14b service placement] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:18:48.368008 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5253823-31ba-4304-80a6-99012c7ce14b service placement] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:18:48.368296 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5253823-31ba-4304-80a6-99012c7ce14b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:18:48.371336 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-a5253823-31ba-4304-80a6-99012c7ce14b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:18:48.387185 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5253823-31ba-4304-80a6-99012c7ce14b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:18:48.389186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5253823-31ba-4304-80a6-99012c7ce14b service placement] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:18:48.400037 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1461/2891] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:18:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:18:48.412843 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7be5c-17c0-488e-97bf-804f66caba3b service placement] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:18:48.486635 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7be5c-17c0-488e-97bf-804f66caba3b service placement] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:18:48.491407 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b35592-0aa0-4bf0-ae16-ae8732c68655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:18:48.492279 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b35592-0aa0-4bf0-ae16-ae8732c68655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:18:48.492642 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b35592-0aa0-4bf0-ae16-ae8732c68655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:18:48.492913 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b35592-0aa0-4bf0-ae16-ae8732c68655 admin admin] PATH_INFO: `/v3/users/c7b128ef63b94a71b7377a08aae7990f` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:18:48.494131 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b35592-0aa0-4bf0-ae16-ae8732c68655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7b128ef63b94a71b7377a08aae7990f)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:18:48.496151 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b35592-0aa0-4bf0-ae16-ae8732c68655 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:18:48.551892 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-53b35592-0aa0-4bf0-ae16-ae8732c68655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7b128ef63b94a71b7377a08aae7990f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:48.552717 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-53b35592-0aa0-4bf0-ae16-ae8732c68655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7b128ef63b94a71b7377a08aae7990f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:48.563139 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-53b35592-0aa0-4bf0-ae16-ae8732c68655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7b128ef63b94a71b7377a08aae7990f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:48.572359 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-53b35592-0aa0-4bf0-ae16-ae8732c68655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7b128ef63b94a71b7377a08aae7990f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:48.573110 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-53b35592-0aa0-4bf0-ae16-ae8732c68655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7b128ef63b94a71b7377a08aae7990f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:48.595653 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1431/2892] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:18:48 2022] DELETE /identity/v3/users/c7b128ef63b94a71b7377a08aae7990f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:18:48.782620 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5253823-31ba-4304-80a6-99012c7ce14b service placement] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:18:48.794637 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5253823-31ba-4304-80a6-99012c7ce14b service placement] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:18:48.853214 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ddded-d264-4f97-9928-64d95d8dac49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:18:48.854085 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ddded-d264-4f97-9928-64d95d8dac49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:18:48.854392 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ddded-d264-4f97-9928-64d95d8dac49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:18:48.854662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ddded-d264-4f97-9928-64d95d8dac49 admin admin] PATH_INFO: `/v3/projects/b1730bdd7de14144bf75c92004923f9a` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:18:48.855801 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ddded-d264-4f97-9928-64d95d8dac49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1730bdd7de14144bf75c92004923f9a)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:18:48.857717 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ddded-d264-4f97-9928-64d95d8dac49 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:18:48.906679 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1c5ddded-d264-4f97-9928-64d95d8dac49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1730bdd7de14144bf75c92004923f9a', 'request_id': 'req-1c5ddded-d264-4f97-9928-64d95d8dac49'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:48.912231 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1c5ddded-d264-4f97-9928-64d95d8dac49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1730bdd7de14144bf75c92004923f9a', 'request_id': 'req-1c5ddded-d264-4f97-9928-64d95d8dac49'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:48.918279 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1c5ddded-d264-4f97-9928-64d95d8dac49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1730bdd7de14144bf75c92004923f9a', 'request_id': 'req-1c5ddded-d264-4f97-9928-64d95d8dac49'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:48.932815 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1c5ddded-d264-4f97-9928-64d95d8dac49 admin admin] The token cache is being invalidate because project b1730bdd7de14144bf75c92004923f9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:18:48.933099 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1c5ddded-d264-4f97-9928-64d95d8dac49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:48.933808 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-1c5ddded-d264-4f97-9928-64d95d8dac49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:48.935488 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1462/2893] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:18:48 2022] DELETE /identity/v3/projects/b1730bdd7de14144bf75c92004923f9a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:18:48.939640 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b35592-0aa0-4bf0-ae16-ae8732c68655 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:18:48.941344 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-53b35592-0aa0-4bf0-ae16-ae8732c68655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:18:48.956092 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b35592-0aa0-4bf0-ae16-ae8732c68655 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:18:49.015783 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21bfe5f-affd-4446-b3f8-a64d69535e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:18:49.016689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bfe5f-affd-4446-b3f8-a64d69535e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:18:49.016971 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bfe5f-affd-4446-b3f8-a64d69535e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:18:49.017240 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bfe5f-affd-4446-b3f8-a64d69535e01 admin admin] PATH_INFO: `/v3/users/0284f5f052214c119531abb71c20dc90` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:18:49.018504 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21bfe5f-affd-4446-b3f8-a64d69535e01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0284f5f052214c119531abb71c20dc90)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:18:49.020696 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21bfe5f-affd-4446-b3f8-a64d69535e01 admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:18:49.022229 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:18:49.075575 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c21bfe5f-affd-4446-b3f8-a64d69535e01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0284f5f052214c119531abb71c20dc90'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:49.076249 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c21bfe5f-affd-4446-b3f8-a64d69535e01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0284f5f052214c119531abb71c20dc90'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:49.084840 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c21bfe5f-affd-4446-b3f8-a64d69535e01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0284f5f052214c119531abb71c20dc90'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:49.091889 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c21bfe5f-affd-4446-b3f8-a64d69535e01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0284f5f052214c119531abb71c20dc90'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:49.092563 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-c21bfe5f-affd-4446-b3f8-a64d69535e01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0284f5f052214c119531abb71c20dc90'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:49.109979 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1432/2894] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:18:48 2022] DELETE /identity/v3/users/0284f5f052214c119531abb71c20dc90 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:18:49.303565 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ddded-d264-4f97-9928-64d95d8dac49 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:18:49.305119 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-1c5ddded-d264-4f97-9928-64d95d8dac49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:18:49.329464 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ddded-d264-4f97-9928-64d95d8dac49 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:18:49.390431 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523db5e-f40d-4524-b113-81e0c8d27650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:18:49.391472 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523db5e-f40d-4524-b113-81e0c8d27650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:18:49.391790 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523db5e-f40d-4524-b113-81e0c8d27650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:18:49.392213 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523db5e-f40d-4524-b113-81e0c8d27650 admin admin] PATH_INFO: `/v3/projects/30f6f1f26604417994fee495603b5838` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:18:49.393623 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523db5e-f40d-4524-b113-81e0c8d27650 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30f6f1f26604417994fee495603b5838)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:18:49.395769 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523db5e-f40d-4524-b113-81e0c8d27650 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:18:49.401980 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:18:49.448595 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4523db5e-f40d-4524-b113-81e0c8d27650 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30f6f1f26604417994fee495603b5838', 'request_id': 'req-4523db5e-f40d-4524-b113-81e0c8d27650'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:49.454286 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4523db5e-f40d-4524-b113-81e0c8d27650 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30f6f1f26604417994fee495603b5838', 'request_id': 'req-4523db5e-f40d-4524-b113-81e0c8d27650'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:49.460438 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4523db5e-f40d-4524-b113-81e0c8d27650 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30f6f1f26604417994fee495603b5838', 'request_id': 'req-4523db5e-f40d-4524-b113-81e0c8d27650'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:49.476328 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4523db5e-f40d-4524-b113-81e0c8d27650 admin admin] The token cache is being invalidate because project 30f6f1f26604417994fee495603b5838 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80124) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:18:49.476533 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4523db5e-f40d-4524-b113-81e0c8d27650 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:49.477291 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4523db5e-f40d-4524-b113-81e0c8d27650 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:18:49.478856 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1463/2895] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:18:49 2022] DELETE /identity/v3/projects/30f6f1f26604417994fee495603b5838 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:18:59.106533 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=26,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:18:59.472356 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:19:26.273405 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21bfe5f-affd-4446-b3f8-a64d69535e01 admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:19:26.275033 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c21bfe5f-affd-4446-b3f8-a64d69535e01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:19:26.440528 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21bfe5f-affd-4446-b3f8-a64d69535e01 admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:19:26.505966 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41137540-2b17-48fa-b532-0aa84c716ea6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474c468a610140f8a5b18d5a1ed1d18e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:19:26.506624 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41137540-2b17-48fa-b532-0aa84c716ea6 service placement] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:19:26.506904 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41137540-2b17-48fa-b532-0aa84c716ea6 service placement] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:19:26.507176 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41137540-2b17-48fa-b532-0aa84c716ea6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:19:26.509280 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-41137540-2b17-48fa-b532-0aa84c716ea6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:19:26.521842 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41137540-2b17-48fa-b532-0aa84c716ea6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:19:26.523371 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41137540-2b17-48fa-b532-0aa84c716ea6 service placement] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:19:26.533752 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1433/2896] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:19:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:19:36.529622 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=89 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:19:48.761845 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523db5e-f40d-4524-b113-81e0c8d27650 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:19:48.763784 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4523db5e-f40d-4524-b113-81e0c8d27650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:19:48.856117 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523db5e-f40d-4524-b113-81e0c8d27650 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:19:48.860320 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aec50b-4ca1-477a-b5c2-ba82e8a5860b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:19:48.861052 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aec50b-4ca1-477a-b5c2-ba82e8a5860b service neutron] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:19:48.861341 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aec50b-4ca1-477a-b5c2-ba82e8a5860b service neutron] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:19:48.861619 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aec50b-4ca1-477a-b5c2-ba82e8a5860b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:19:48.863618 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b6aec50b-4ca1-477a-b5c2-ba82e8a5860b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:19:48.923662 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aec50b-4ca1-477a-b5c2-ba82e8a5860b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:19:48.925164 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aec50b-4ca1-477a-b5c2-ba82e8a5860b service neutron] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:19:48.934351 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1464/2897] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:19:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:19:58.930011 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=59 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:20:02.149080 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41137540-2b17-48fa-b532-0aa84c716ea6 service placement] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:20:02.150506 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-41137540-2b17-48fa-b532-0aa84c716ea6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:20:02.162293 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41137540-2b17-48fa-b532-0aa84c716ea6 service placement] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:20:02.165939 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b59cd-2fa8-4740-b4a9-f0a1d470f2b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaffb9f0bf714822829455c0487376ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:20:02.166574 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b59cd-2fa8-4740-b4a9-f0a1d470f2b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:20:02.166852 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b59cd-2fa8-4740-b4a9-f0a1d470f2b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:20:02.167128 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b59cd-2fa8-4740-b4a9-f0a1d470f2b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:20:02.168981 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-c68b59cd-2fa8-4740-b4a9-f0a1d470f2b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:20:02.261945 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b59cd-2fa8-4740-b4a9-f0a1d470f2b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:20:02.263325 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b59cd-2fa8-4740-b4a9-f0a1d470f2b8 service neutron] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:20:02.282776 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1434/2898] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:20:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:20:08.262370 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aec50b-4ca1-477a-b5c2-ba82e8a5860b service neutron] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:20:08.264190 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-b6aec50b-4ca1-477a-b5c2-ba82e8a5860b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:20:08.279629 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aec50b-4ca1-477a-b5c2-ba82e8a5860b service neutron] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:20:08.284383 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9aff42-9a6d-46d5-97fa-e59cd6642d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:20:08.285358 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9aff42-9a6d-46d5-97fa-e59cd6642d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:20:08.285653 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9aff42-9a6d-46d5-97fa-e59cd6642d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:20:08.285933 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9aff42-9a6d-46d5-97fa-e59cd6642d32 admin admin] PATH_INFO: `/v3/users/0ef599055aed42f3a97fb30bba553fb0` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:20:08.287251 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9aff42-9a6d-46d5-97fa-e59cd6642d32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ef599055aed42f3a97fb30bba553fb0)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:20:08.289244 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9aff42-9a6d-46d5-97fa-e59cd6642d32 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:20:08.339771 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4e9aff42-9a6d-46d5-97fa-e59cd6642d32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ef599055aed42f3a97fb30bba553fb0'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:08.347186 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4e9aff42-9a6d-46d5-97fa-e59cd6642d32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ef599055aed42f3a97fb30bba553fb0'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:08.347899 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4e9aff42-9a6d-46d5-97fa-e59cd6642d32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ef599055aed42f3a97fb30bba553fb0'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:08.357039 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4e9aff42-9a6d-46d5-97fa-e59cd6642d32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ef599055aed42f3a97fb30bba553fb0'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:08.357683 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-4e9aff42-9a6d-46d5-97fa-e59cd6642d32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ef599055aed42f3a97fb30bba553fb0'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:08.376011 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1465/2899] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:20:08 2022] DELETE /identity/v3/users/0ef599055aed42f3a97fb30bba553fb0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:20:08.556839 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b59cd-2fa8-4740-b4a9-f0a1d470f2b8 service neutron] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:20:08.569075 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b59cd-2fa8-4740-b4a9-f0a1d470f2b8 service neutron] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:20:08.631520 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a390964-c63c-4848-86bf-720ae8d0930f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:20:08.632260 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a390964-c63c-4848-86bf-720ae8d0930f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:20:08.632566 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a390964-c63c-4848-86bf-720ae8d0930f admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:20:08.632837 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a390964-c63c-4848-86bf-720ae8d0930f admin admin] PATH_INFO: `/v3/projects/c5bb30b042f3422faf43af82b27b37a5` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:20:08.634022 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a390964-c63c-4848-86bf-720ae8d0930f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5bb30b042f3422faf43af82b27b37a5)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:20:08.635919 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a390964-c63c-4848-86bf-720ae8d0930f admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:20:08.685338 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-4a390964-c63c-4848-86bf-720ae8d0930f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5bb30b042f3422faf43af82b27b37a5', 'request_id': 'req-4a390964-c63c-4848-86bf-720ae8d0930f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:08.691541 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-4a390964-c63c-4848-86bf-720ae8d0930f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5bb30b042f3422faf43af82b27b37a5', 'request_id': 'req-4a390964-c63c-4848-86bf-720ae8d0930f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:08.697959 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-4a390964-c63c-4848-86bf-720ae8d0930f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5bb30b042f3422faf43af82b27b37a5', 'request_id': 'req-4a390964-c63c-4848-86bf-720ae8d0930f'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:08.701770 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:20:08.716111 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-4a390964-c63c-4848-86bf-720ae8d0930f admin admin] The token cache is being invalidate because project c5bb30b042f3422faf43af82b27b37a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:20:08.716395 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-4a390964-c63c-4848-86bf-720ae8d0930f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:08.717096 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-4a390964-c63c-4848-86bf-720ae8d0930f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:08.718689 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1435/2900] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:20:08 2022] DELETE /identity/v3/projects/c5bb30b042f3422faf43af82b27b37a5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:20:08.723015 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9aff42-9a6d-46d5-97fa-e59cd6642d32 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:20:08.724680 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-4e9aff42-9a6d-46d5-97fa-e59cd6642d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:20:08.740184 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9aff42-9a6d-46d5-97fa-e59cd6642d32 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:20:08.800675 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ef9fb-8e56-425a-bfe3-0512425f0150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:20:08.801471 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ef9fb-8e56-425a-bfe3-0512425f0150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:20:08.801753 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ef9fb-8e56-425a-bfe3-0512425f0150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:20:08.802028 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ef9fb-8e56-425a-bfe3-0512425f0150 admin admin] PATH_INFO: `/v3/users/621480ae82d840a3927d879cbc375d7c` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:20:08.803453 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ef9fb-8e56-425a-bfe3-0512425f0150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=621480ae82d840a3927d879cbc375d7c)` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:20:08.805434 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ef9fb-8e56-425a-bfe3-0512425f0150 admin admin] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:20:08.856040 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-178ef9fb-8e56-425a-bfe3-0512425f0150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '621480ae82d840a3927d879cbc375d7c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:08.863641 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-178ef9fb-8e56-425a-bfe3-0512425f0150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '621480ae82d840a3927d879cbc375d7c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:08.864388 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-178ef9fb-8e56-425a-bfe3-0512425f0150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '621480ae82d840a3927d879cbc375d7c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:08.873789 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-178ef9fb-8e56-425a-bfe3-0512425f0150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '621480ae82d840a3927d879cbc375d7c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:08.874592 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.notifications [None req-178ef9fb-8e56-425a-bfe3-0512425f0150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '621480ae82d840a3927d879cbc375d7c'} {{(pid=80124) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:08.883105 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:20:08.895766 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1466/2901] 10.222.0.13 () {58 vars in 1223 bytes} [Sat Jun 25 16:20:08 2022] DELETE /identity/v3/users/621480ae82d840a3927d879cbc375d7c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:20:09.077830 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a390964-c63c-4848-86bf-720ae8d0930f admin admin] Authenticating user token {{(pid=80125) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:20:09.079421 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.fernet_utils [None req-4a390964-c63c-4848-86bf-720ae8d0930f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80125) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:20:09.093931 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a390964-c63c-4848-86bf-720ae8d0930f admin admin] Validating token access rules against request {{(pid=80125) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:20:09.151456 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af4ca6-0196-4a89-a61e-909410d886ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c132d4e24cfd4bd197338f840d4d9eb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c8597e0eb76849bdadd0530f7e198ab0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80125) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:20:09.152316 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af4ca6-0196-4a89-a61e-909410d886ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:20:09.152626 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af4ca6-0196-4a89-a61e-909410d886ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:20:09.152901 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af4ca6-0196-4a89-a61e-909410d886ad admin admin] PATH_INFO: `/v3/projects/cabbc5fd45484def9cc8f536d576d517` {{(pid=80125) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:20:09.154147 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af4ca6-0196-4a89-a61e-909410d886ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=cabbc5fd45484def9cc8f536d576d517)` {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:20:09.156091 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af4ca6-0196-4a89-a61e-909410d886ad admin admin] RBAC: Authorization granted {{(pid=80125) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:20:09.204527 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-67af4ca6-0196-4a89-a61e-909410d886ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cabbc5fd45484def9cc8f536d576d517', 'request_id': 'req-67af4ca6-0196-4a89-a61e-909410d886ad'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:09.210816 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-67af4ca6-0196-4a89-a61e-909410d886ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cabbc5fd45484def9cc8f536d576d517', 'request_id': 'req-67af4ca6-0196-4a89-a61e-909410d886ad'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:09.216856 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-67af4ca6-0196-4a89-a61e-909410d886ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cabbc5fd45484def9cc8f536d576d517', 'request_id': 'req-67af4ca6-0196-4a89-a61e-909410d886ad'} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:09.231313 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-67af4ca6-0196-4a89-a61e-909410d886ad admin admin] The token cache is being invalidate because project cabbc5fd45484def9cc8f536d576d517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80125) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 25 16:20:09.231621 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-67af4ca6-0196-4a89-a61e-909410d886ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:09.232256 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG keystone.notifications [None req-67af4ca6-0196-4a89-a61e-909410d886ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80125) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 25 16:20:09.233800 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: [pid: 80125|app: 0|req: 1436/2902] 10.222.0.13 () {58 vars in 1232 bytes} [Sat Jun 25 16:20:09 2022] DELETE /identity/v3/projects/cabbc5fd45484def9cc8f536d576d517 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 25 16:20:18.895015 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:20:19.228496 dv-cinder-847620-2-iscsi devstack@keystone.service[80125]: DEBUG dbcounter [-] [80125] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80125) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 25 16:24:28.277185 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ef9fb-8e56-425a-bfe3-0512425f0150 admin admin] Authenticating user token {{(pid=80124) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 25 16:24:28.279166 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-178ef9fb-8e56-425a-bfe3-0512425f0150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:24:28.362796 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ef9fb-8e56-425a-bfe3-0512425f0150 admin admin] Validating token access rules against request {{(pid=80124) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 25 16:24:28.430798 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e9933-fdf4-406f-a8e0-1f66bc6d535c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '474c468a610140f8a5b18d5a1ed1d18e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3600648eb94b434b8c797d2f454e46b8', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80124) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 25 16:24:28.431446 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e9933-fdf4-406f-a8e0-1f66bc6d535c service placement] REQUEST_METHOD: `GET` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 25 16:24:28.431729 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e9933-fdf4-406f-a8e0-1f66bc6d535c service placement] SCRIPT_NAME: `/identity` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 25 16:24:28.432010 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e9933-fdf4-406f-a8e0-1f66bc6d535c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80124) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 25 16:24:28.434427 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.fernet_utils [None req-923e9933-fdf4-406f-a8e0-1f66bc6d535c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80124) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 25 16:24:28.446725 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e9933-fdf4-406f-a8e0-1f66bc6d535c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 25 16:24:28.448329 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e9933-fdf4-406f-a8e0-1f66bc6d535c service placement] RBAC: Authorization granted {{(pid=80124) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 25 16:24:28.459120 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: [pid: 80124|app: 0|req: 1467/2903] 10.222.0.13 () {60 vars in 1365 bytes} [Sat Jun 25 16:24:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 25 16:24:38.454991 dv-cinder-847620-2-iscsi devstack@keystone.service[80124]: DEBUG dbcounter [-] [80124] Writing DB stats keystone:SELECT=58 {{(pid=80124) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}